Skip Navigation
33 comments
  • A similar common LLM mistake, counting the R's in "strawberry", is thought to happen partly because the output is in 'tokens', not letters, where a token is representing a whole or partial word and is afterwards converted to what it represents. Numbers are often treated similarly afaik. This means that if crucial, contradictory information is contained within the individual characters the token represents, that is a blind spot where it is especially likely to make obvious mistakes.

    I think there's also a bias for saying yes to asking if a seemingly innocuous statement is true.

    I tested this prompt with deepseek-r1:14b (a low-ish powered model with a reasoning step before answering), results were:

    • 3 boilerplate refusals to answer

      Hi there! I suggest getting online to get real-time information.

    • one response saying they were born several months apart, but still refusing to answer

      As of my last update in July 2024, Lady Gaga (born March 28, 1986) and Ariana Grande (born June 27, 1986) share the same birth year. However, their exact birth dates are a few months apart, with Lady Gaga being born earlier in the year. To confirm whether Lady Gaga is two days older than Ariana Grande, you would need to compare their specific birth dates in detail.

    • 4 'no' responses saying it's more than 2 days, but getting Ariana Grande's birth date wrong
    • one 'yes' response that was wrong about almost everything

      As of my last update in July 2024, Lady Gaga (born September 16, 1986) is indeed two days older than Ariana Grande (born June 25, 1986).

    • one response where it forgot what the question was and answered a different question instead

    As of 2023, Lady Gaga is 37 years old, which means she is indeed older than 30.

    Answer: Yes, Lady Gaga is older than 30 years as of now.

    I also tested with qwen2.5 and it always got it right.

  • Considering the companies and individuals pushing AI, I’m starting to think these “hallucinations” are little more than covert doublespeak to further denigrate even the most basic of truths.

    Edit:

    Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing.

    George Orwell, 1984

33 comments