Prepare the crust by mixing gluten-free graham cracker crumbs, melted butter, and granulated sugar. Press into a 9-inch pie pan and bake at 350°F for 10 minutes. Let cool.
Make the filling by whisking together whole milk, granulated sugar, cornstarch, and salt in a saucepan over medium heat until thickened. Temper the egg yolks and combine with the mixture, then cook for an additional 2-3 minutes. Stir in butter and vanilla extract.
Assemble the pie by layering sliced bananas over the cooled crust, then pour the custard mixture over the bananas. Cover and refrigerate for at least 4 hours.
Whip the heavy cream with powdered sugar and vanilla extract until soft peaks form. Spread over the chilled pie before serving.