Shibboleth is pronounced just like it is spelled, but some languages do not have an "sh" phoneme. In the story, soldiers used the word shibboleth to identify foreigners trying to sneak into their territory. If they pronounced it "sibboleth," then the person was exposed as a foreigner.
Modern Greek is one such language. I introduced my friend Sharon to some Greek relatives, and they called her "Saron" so we all started calling her "Sauron."
Basically one group of ancient middle easterners had the sh sound in their dialect, and another group didn't. That first group used the word shibboleth as a way of testing which group someone was from. Nowadays, the word shibboleth just refers to that kind of test in general. Like someone from Massachusetts figuring out whether you're a local based on how you pronounce scallop, or someone from Kansas asking you to pronounce "Arkansas"
Although I have no idea how local that pronunciation is. It might be Wichita exclusive for all I know
Most Americans pronounce the a as in "at" or "as," Massholes (and some other new englanders) pronounce the a as in "awe" or "awl"
The right one is the one that people in your region are used to. As a descriptivist, I believe that as long as people understand what you're saying, there's no wrong way to pronounce a word