[A nod.] It... does seem likely. It would be nice, I think, to be able to remember, but... with the way things seem to work, it's hard to imagine being allowed to. If the people later on the timelines haven't noticed anything amiss or even heard anything about Asgard....

[Then that kind of implies that either memories for those earlier were wiped or that literally everyone suddenly became excellent at hiding things.]