In 1887, priest Juozapas Ambrazevičius and the parishioners built a new two-tower brick church which was given the name of St. Joseph. Bells were loaded into one of the towers. Organ was brought from the old church and installed. There was a cemetery chapel in the castle cemetery. During the World War II, the church burned down, the towers were demolished. And in the post-war years it was rebuilt but with lower towers. The modern interior of the church is dominated by classical space, combined with Gothic traditions.