We have a room like that. We insulated the windows as everyone else has already suggested, put a thick rug down on the floor, but there were still strong drafts that were coming from the outlets and the baseboards. We put those baby-proofing plugs in any unused outlets, weather-taped around the covers, and then caulked the baseboards. If you can't manage caulk atm just use hot glue. It will peel up nicely in the summer when you are ready to caulk.

Make sure to use a humidifier, especially if you get a space heater.

We also found using a higher bed and not putting anything underneath it so that air could flow under the bed and move around the room helped a lot.