In a small saucepan, whisk together the milk, pumpkin puree, pumpkin pie spice, and cinnamon. Place over medium heat and cook, stirring occasionally, until starting to simmer. Remove from heat, whisk in the vanilla extract, and allow to cool.
In a blender, pulse together the cooled milk pumpkin mixture, ice cream, and ice cubes until smooth.
Transfer the mixture among serving glasses.
Top each milkshake with whipped cream, caramel sauce, graham crackers, decorating sugar or sprinkles, and an additional pinch of pumpkin pie spice if desired.