Land or Water?

Andrej Karpathy on X: "Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet."

Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.

Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.

fun detail: a model that just answers "water" every time scores roughly 70% here, since most of the planet is ocean. so Sonnet 4.5 at 60% is actually worse than never guessing land. 90%+ is where it really starts knowing the map