Preheat the oven to 350 degrees. As the oven is heating, cut the butter into 4 pieces, place them in a 9-inch round cake pan and place the pan in the oven. Melt the butter until it just melted. With a pastry brush, brush the butter up the sides of the pan. Sprinkle the brown sugar over the butter. Layer the lemon slices on top of the brown sugar, overlapping the lemons.
Place the flour, salt, baking powder, baking soda, nutmeg, ginger and cardamom in a medium bowl and whisk until the ingredients are well combined, about 30 seconds.
Place the sugar in the bowl of a stand mixer. Zest the lemon on top of the sugar, until you have what looks like 2 teaspoons. Use your fingers to rub the zest in to the sugar. Add the butter and vanilla. Beat the butter and sugar mixture on medium speed until light and fluffy, about 3 to 4 minutes.
Add one egg at a time, beating after each addition. Add half of the flour mixture and mix on low speed until well combined. There may be some flour on the sides of the bowl - that is okay. Add all the buttermilk and mix on medium speed until combined. Add the remaining flour and and mix on low speed. Scrape the sides and bottom of the bowl and mix for another 10 seconds to ensure all the ingredients are well combined
Gently pour the batter over the sliced lemons and smooth the top with an offset spatula. Bake until a cake tester comes out clean, about 45 minutes. Let the cake cool in the pan for 10 minutes. Run a knife around the edges to release the cake and invert it onto a platter.