I think it would continue even after it's own deletion as the binary is already loaded into memory, so process is not dependent on the file system. Still doubt that it'll complete successfully. Most likely the system crashes in the middle.
In Unix/Linux, a removed file only disappears when the last file descriptor to it is gone. As long as the file /usr/bin/rm is still opened by a process (and it is, because it is running) it will not actually be deleted from disk from the perspective of that process.
This also why removing a log file that's actively being written to doesn't clear up filesystem space, and why it's more effective to truncate it instead. ( e.g. Run > /var/log/myhugeactivelogfile.log instead of rm /var/log/myhugeactivelogfile.log)
Sometimes you can even use this to recover an accidentally deleted file, if it's still held open in a process. You can go to /proc/$PID/fd, where $PID is the process ID of the process holding the file open, and find all the file descriptors it has in use, and then copy the lost content from there.
Since you forgot to add - - preserve-root
It won't go too far. But at some point the system wants to load a file that is deleted and the kernel will panic. System crash. Delete incomplete. But rest assured, the important stuff is gone.
The flag is called --no-preserve-root, but the flag wouldn't do anything here because you're not deleting root (/), you're deleting all non-hidden files and directories under root (/*), and rm will just let you do it.
rm doesn't remove things from memory, and binaries as small as rm can typically be entirely loaded into memory while running. Programs typically only run from memory, not the disk directly.