This gets complicated. The small heater is almost certain to be resistive electric heat. Your central heat could be anything. In my area, the gas furnace is usually cheaper to run for the entire house than it is for a space heater in a single room.
OTOH, if you have a resistive electric furnace, your advice is spot on
Electric heaters may actually cost more in electric bills than you would spend on heating. It depends on the type of heating and the size of your space, but something like natural gas might cost less to keep the whole house at 65 degrees, rather than trying to warm a single room from 45 degrees with a space heater.