Preheat your oven to 350°F (175°C) and grease and flour a 9-inch round cake pan.
Mash the ripe bananas in a bowl until smooth.
In a large mixing bowl, cream together the softened butter, granulated sugar, and brown sugar until light and fluffy.
Add the mashed bananas, eggs, and vanilla extract to the butter mixture and mix until well combined.
In a separate bowl, whisk together the gluten-free flour, baking powder, baking soda, and salt. Gradually add this dry mixture to the wet ingredients, alternating with the buttermilk.
Pour the batter into the prepared cake pan and smooth the top. Bake for 25-30 minutes, checking for doneness with a toothpick.
Let the cake cool in the pan for about 10 minutes before transferring it to a wire rack to cool completely.
Frost with a simple glaze made from powdered sugar and milk, or enjoy it plain.