I would walk around the city center (from Bismarkplatz along Hauptstrasse), to see the shops, the buildings, the churchs. The atmosphere is great. There is a lovely walking path (stairs) to reach the castle, though entering the castle is not much worth it. You can take the cable-car to reach "König Stühl" at the top, to have great views of the city. You can make all that one day, and if you like hiking, there's something I wouldn't miss: Take the way to Philosophenweg, and walk up the mountain on the other side of the river (I mean in relation with the old town and the castle), and follow the paths that are shown in the map to reach "Thingstätte" and the ruins of a very old closter. You will love it! It's awesome and not full of tourists!

Don't miss a stroll along the main street, the castle (in the late afternoon you enter for free and it is easy to reach thanks to a cable-car) and, if the weather is nice and you feel like to walk, the Philosophenweg. Have a nice w-e!