The point of my second statement is that if you made an AI that stores and retrieves phone numbers that the model could reasonable use phone number chunks in its random number generation. A phone number can normally be broken into 3 to 6 chunks of 1 to 5 numbers which is reasonable sizes to tokenize. If you then asked it for a random number I think it is reasonable that it would be as likely if not more likely to use the data from the phone number list as it would to use the core 0 to 9 tokenized number list unless you specifically tried to split the two.
This is a WhatsApp AI so I think asking it for Tim's number is a use case they trained on. It needs to be a phone book. My guess is they said that list A is a list of public numbers for training things like what a phone number looks like, and list B is a list of private user numbers. Now while a random number could be a random string of numbers it could also be that the LLM is too likely to pull a combination that is actually a real number.
So is this a case where it randomly pulled together 11 digits that magically hit the roughly 1 in in 100 chance that a random string of numbers shaped like a UK phone number would be a number of a user. Is it a case where it pulled from a public combo list of 4 tokens and randomly reformed a real number that was both public and private? That seems more likely to me. We probably won't ever get to know.
If I was making this AI chat bot I would have it check against the most critical data I have for privacy before it shared it as a random number though. WhatsApp phone numbers are its users IDs. Even if it truly randomly generates one it should verify that it is a private number and not output it as it showed it could do when questioned where the number came from.
