Jaffna is definitely worth a visit. The city is not over sized yet so you can still wander or cycle around without too much hassle. Besides a beautiful Hindu temple, the Dutch fort and the rebuilt library, the city doesn't have so many sightseeing options. Just strolling around, having a cup of tea and catch the vibe is already enough.
Close to Jaffna you can visit some islands. Most of them are connected with each other, which makes it ideal to rent a scooter and cruise around. Options enough to have a pit stop at the many temples you come across or like we did, have a refreshing swim at one of the many beautiful beaches. When cruising around we saw many abandoned and destroyed houses. A lot of families fled their houses during the war and never returned. We felt right in the middle of a ghost town.
The furthest island, Delft, can only be visited by ferry as it is not connected to the peninsula of Jaffna. You can still find some remains of the Dutch colonial time here.
The food here is influenced by South-India. You can still find the rice and curry but it is served with a different kind of rice, named Thali. We also discovered dosa's here, a nice change of diet.
In short, Jaffna deserves a spot on your itinerary. Although, let's hope it doesn't get spoiled by mass tourism.