Broadly speaking, Japan's principle sights can be divided into temples, gardens, castles, volcanoes (of which there are very many of each) and everything else.
Temples
Temples are generally Shinto or Buddhist. Shinto temples are usually quite ascetic and in most of the important ones (like the Ise Jingu complex) you cannot enter even the outer walls. They are preceded by a Torii gate (see below) and involve a few rituals which are easily followed, if you wish, by observing local worshippers.
Part of the Ise Jingu complex
The most photographed is the Itsukushima shrine with its floating Torii gate on Miyajima (the JR Pass is good for the ferry across from the mainland).
Itsukushima shrine with Torii in foreground
Buddhist temples, in contrast, are grander and provide greater access for the visitor. They range from the spectacular Senso-Ji in Tokyo and any number in Kyoto to the huge Todai-Ji in Nara and the ancient Shitennoji in Osaka.
Senso-Ji, Tokyo
Todai-Ji, Nara
Kinkaku-ji is a Zen Buddhist temple in Kyoto
Religious complexes
There are plenty to choose from. Nikko and Koya are large other worldly religious complexes.
Part of the Koya complex
Part of the Nikko complex
Gardens
There are gardens absolutely everywhere but being Japan there are the most highly regarded standout ones, others which are just as good and the rest. The most highly regarded ones are Kenroku-en at Kanazawa (superb and nowadays very crowded), Kora Kuen at Okayama (very classic) and Kaira Koen at Mito (smaller and quieter and most notable for its plum blossom in early spring). Personally I think the Shinjuku-Gyoen National Garden in Tokyo, the Kyoto Gyoen National Garden, Suizenji-Jojuen at Kumamoto and Ritsurin at Takamatsu are as good but you could say that about plenty of others including the small and tradtional Rikugien in Tokyo. Many castles and temples have gardens of varying appeal too. If you have a couple of hours to kill in Tokyo then Hamarikyū gardens near Shiodome are interesting as they, uniquely, have salt water lakes and water ways.
Kenroku-en, Kanazawa
Kora Kuen, Okayama
Sengan-en near Kagoshima is interesting as it is one of the most southerly (of the home islands) so has a different mix of plants and it is co-located with an industrial museum as the area was at the forefront of Japanese industrialisation.
Sengan-en
Tofuku-Ji is a temple garden in Kyoto and is notable as it was celebrated garden designer, Mirei Shigemori's, first major work.
The moss garden at Tofuku-ji
Castles
Again being Japanese there are a few that are most highly regarded and the rest. The design is broadly similar with the number of stories being the single most significant differentiator. Built of wood they have burnt/fallen down and rebuilt many, many times sometimes in wood (Himeji, Matsumoto and Matsue, three of the five 'national treasure' castles) and sometimes in concrete such as Hiroshima, Aizu & Wakayama – OK as museums or Osaka - magnificent with extensive double moated grounds. The concrete castles are easier to navigate as they have wider and less steep stairs. Others such as the delightful Kumomoto, right in the centre of town, are being rebuilt.
Himeji shortly after its re-opening
Matsumoto
Okayama
A word or warning. Visiting castles often involves lengthy queues and climbing up and down many flights of steep, narrow, highly polished and crowded stairs usually in plastic slippers five sizes too small!
Watch out for the steep stairs in many castles
Volcanoes
From Mount Shari on the Shirotoko peninsula in Northern Hokkaido to the smouldering Sakurajima opposite Kagoshima in Southern Kyushu there are volcanoes to be spied almost everywhere in Japan. Of course it is Mt Fuji that is the star attraction and, on a clear day, there aren’t many better views than from the Tokaido Shinkansen between Odawara and Shizuoka. First time visitors are usually encouraged to take the Hakone Tour which involves train, ropeway (funicular), bus and boat journeys as well as a bit of hiking around the Fuji region. We last did it many years ago and it was very crowded then with a lot of queueing and felt like a test of endurance. It may have improved since then but maybe not...... Get to Lake Yamanaka, you are much closer and the views are spectacular.
Mt Yotei on Hokkaido
Many volcanoes have ropeways which provide good access to higher altitude hiking. One surprising feature of these is that they are most unlike the other shiny, modern and advanced transport systems that abound in Japan. They look like rather scruffy second hand imports from elsewhere in the world and while undoubtedly safe they strike an odd note. Be aware, though, that volcanoes attract clouds and are often obscured.
Mt Sakurajima across the bay from Kagoshima
Many of the volcanoes are active and are sometimes closed for long periods such as Mt Aso on Kyushu. There are also plenty of thermal areas and crater lakes to enjoy. Note that even though there may not be any eruptions taking place roads can be closed due to the presence of toxic fumes. Jigokudani (Hell Valley) near Noboribetsu, Hokkaido and the Kirishima area in the south of Kyushu are notable thermal areas worth visiting.
Asahi-dake reached from the ropeway at Daisetsuzan
Although not predominantly volcanic the Japanese Alps in Nagano prefecture are well worth a visit and good views are possible from Toyama all the way round to Matsumoto.
The Japanese Alps from Toyama
Most of the better known volcanoes and mountainous areas are situated in some of Japan's 30 or so National Parks which also include coastal areas and wetlands.