The Church was restored (and indeed, pretty well rebuilt) by the architectural company for whom Hardy worked in 1860. 

Hardy himself had his doubts about the kind of church "restoration" that the Victorians went in for.  In the case of St Juliot, it was clearly excusable.  However often the Victorians' enthusiasm resulted in the combined architectural changes of centuries being swept away.  The churches may have been pretty, but they had lost their soul. 

"Twenty-one years and six months do not pass without setting a mark even upon durable stone and triple brass; upon humanity such a period works nothing less than transformation. In Barnet's old birthplace vivacious young children with bones like india-rubber had grown up to be stable men and women, men and women had dried in the skin, stiffened, withered, and sunk into decrepitude; while selections from every class had been consigned to the outlying cemetery. Of inorganic differences the greatest was that a railway had invaded the town, tying it on to a main line at a junction a dozen miles off. Barnet's house on the harbour-road, once so insistently new, had acquired a respectable mellowness, with ivy, Virginia creepers, lichens, damp patches, and even constitutional infirmities of its own like its elder fellows. Its architecture, once so very improved and modern, had already become stale in style, without having reached the dignity of being old-fashioned. Trees about the harbour-road had increased in circumference or disappeared under the saw; while the church had had such a tremendous practical joke played upon it by some facetious restorer or other as to be scarce recognizable by its dearest old friends."

You can find information on St Mary's Bridport at the attached website: 

http://www.westgallerychurches.com/Dorset/Bridport/Bridport.html