You would not part an old man from his walking stick?

I love how Wormtongue regresses to an angry 14 year old...

"Awww... I TOLD YOU to take the wizard's STAFF! GOSH!"

Thing is though, as Gandalf the White, he likely didn't likely NEED the staff to eject Saruman's influence, it just made it easier.