Others did more intricate work, tenderly wiping dirt off Buddhist statues and stone carvings.
It's not the way out-of-towners normally spend the start of the so-called Golden Week holiday, when Japanese commonly leave big cities to visit their home towns, take hot spring vacations or travel abroad. But after last month's earthquake and tsunami decimated northeastern coastal towns and left an estimated 26,000 Japanese either dead or missing, these are not normal times.
"I saw the devastation on TV and felt I had to do something," said Junko Sugino, 49, as she dragged a crate of mud through the narrow lanes between the tombstones.
"This is hard work, but it's something that has to be done by people. Machines can't fit into these tiny spaces," she said.
Sugino, from the western city of Nara, is among tens of thousands of helpers expected to converge on Japan's northeast in coming days.
At hard-hit Ishinomaki city's Senshu University, which has become one of the region's largest volunteer centers, administrators have been so deluged by inquiries they've started telling applicants to stay home or postpone their trip until after Golden Week.
Some 1,500 volunteers already are camped on the university's sports fields, Ishinomaki welfare department manager Katsuhito Ito said.
Farther north, in Iwate Prefecture, officials are bracing for an influx of volunteers on four-day tours organized by travel agencies through May 8.
They're paying 19,000 yen ($232) for bus fare, accommodations and the opportunity to remove rubble from homes in the cities of Yamada, Otsuchi and Noda, said Iwate official Susumu Sugawara.
Noriyuki Owaki, 37, another of the workers at the Jionin temple cemetery said he's never volunteered for anything before, but decided almost immediately after the March 11 disaster that he would help out during Golden Week.
"It's meaningful work, because you're dealing with so many families' memories," Oikawa said of his cemetery toils.
Also on Friday, at Tokyo's Gokokuji temple, the Dalai Lama presided over prayers for victims of the twin disasters. The spiritual leader of Tibetan Buddhists said that despite the suffering and destruction there is "no reason to feel discouraged and remain hopeless."
While Japanese communities have long had a tradition of looking out for one another, organized non-profit-backed volunteer groups who parachute into trouble spots are relatively new.
The 1995 earthquake in the city of Kobe was a watershed moment for volunteerism in Japan, said Charles McJilton, founder the Second Harvest Japan national food bank.
Many people wanted to help Kobe victims, but the government was unable to handle the influx of volunteers. That experience led to a new law on nonprofit organizations in 1998 that allowed citizens to incorporate as legal entities, McJilton said.
"There hadn't been a history of volunteerism, but there's a tremendous surge of interest in volunteering right now," said David Campbell, who directs the U.S.-based nonprofit All Hands Volunteers.
In Japan, a string of national holidays at the end of April and beginning of May are collectively called Golden Week. If there's a downside to the Golden Week volunteer boom, it's being felt by the traditional tourism industry, which usually cashes in on holiday business.
JTB Corp., the country's largest travel agency, forecast that people traveling domestically between April 24 and May 5 would drop some 28 percent from the previous year, while travelers abroad would sink nearly 17 percent.
One tourism industry advertising campaign is urging Japanese to visit the damaged Tohuku region, saying that their spending will help with the area's recovery, saying "Tohoku's path to recovery may be long and difficult, but we want tourism, one of the region's main industries, to be a bright spot along the way."
Prime Minister Naoto Kan has even urged Japanese to open their wallets during the holiday to help prod the post-disaster economy.
But Toshinobu Muto, director of the Tokyo-based Fareast Inc. travel agency said those pleas will likely fall on deaf ears.
"The Japanese have a custom where if their neighbors have it really bad, they try to be quiet, so that kind of mindset makes a lot of people really not want to travel," Muto said.
Back at the Ishinomoki volunteer center, which has hosted more than 27,000 helpers since March 15, lead coordinator Hideo Otsuki was grateful for the swing from traditional vacations to volunteerism, but wondered how long it would last.
"There is a concern that volunteers may stop coming after Golden Week," he said. "We hope that's not the case because we need them."