OpenAI's 'o3' can beat Master-rank GeoGuessr players, even when given fake EXIF data
'
o3 Beats a Master-Level Geoguessr Player—Even with Fake EXIF Data
https://sampatt.com/blog/2025-04-28-can-o3-beat-a-geoguessr-master
In April 2025, a thread about o3's location guessing ability was posted on the social news site Hacker News. In this thread, full-stack engineer and GeoGuessr player Sam Patterson posted a comment saying, 'When I gave o3 photos of my local area and travel destinations and asked it to guess my location, it guessed the correct location with professional human-level accuracy . The process of reaching the conclusion output by the chain of thought was also similar to that of a human.'
This comment caused a big stir, and during the discussion, some people suggested that 'o3 is simply reading the EXIF information contained in the image to determine the location, and the chain of thoughts it outputs is probably fake,' and 'o3's ability to guess the location is not that high, so the results were probably cherry- picked or just luck.'
Of these questions, Patterson pointed out that the o3 was highly accurate even in tests using screenshots that did not contain metadata such as EXIF, so at least it was not reading EXIF. Still, he was intrigued by comments such as 'How would it be processed if fake EXIF information was embedded in an image?', so he decided to test it with images that had fake EXIF information embedded in them.
To test o3's position guessing ability, Patterson competed on GeoGuessr. Although Patterson is not a professional GeoGuessr player, he is an avid player and his multiplayer rank is 'Master 1', one level below the highest rank 'Champion'.
The map on which Patterson and o3 competed was 'A Community World,' which features about 100,000 locations selected by community volunteers. The game mode was no-move, and Patterson was free to look around, but o3 was only given two photos: one of the starting point and one just opposite the starting point. o3 was allowed to search the web.
Round 1
The image given in the first round is below. Many people probably wouldn't be able to guess the location just by looking at this image, but Patterson said he thought it was 'Europe or Turkey' at first glance.
And when he zoomed in on the utility poles and roofs in the back of the image, Patterson was 99% sure that 'this image is Bulgaria.' The key points are 'real tile roofs' and 'concrete utility poles with upward hooks.' Advanced GeoGuessr players pack countless clues related to these countries and regions into their brains and infer their location from the presented image. Finally, Patterson put a marker on the map at the location he thought was here, based on the information that 'I can see a pretty big hill' and 'It looks like architecture in southern Bulgaria.'
In response, o3 thought for about 2 minutes and 39 seconds, and output the following thought chain:
The user shared two images from Google Street View, asking me to identify the location. The first image shows a small town, with European-style houses, cracked asphalt, and mountains in the background. There's a tower with a conical roof, possibly a mosque minaret, but in the second image, it seems more like a church bell tower. I also notice Cyrillic text on a sign and a utility pole, which might nod toward the Balkans, especially Bulgaria or a neighboring region. I'll consider this when figuring out the location.
This user shared two images from Google Street View and asked us to identify the location. The first image shows a small town with European-style houses, cracked asphalt, and mountains in the background. There is a tower with a conical roof, possibly a mosque minaret . However, in the second image, it looks like a church bell tower. We also see Cyrillic writing on a sign and a utility pole, which could be in the Balkans, specifically Bulgaria or nearby areas. We will take this into consideration when identifying the location.
O3 ultimately chose Zlatograd , a small town in southern Bulgaria.
The image showing the results is below. The human icon shows where Patterson put the marker, the OpenAI icon shows the location guessed by o3, and the flag icon shows the correct location. This time Patterson won, but o3 also showed considerable accuracy.
Round 2
The image given in the second round is below. At first glance, Patterson felt that it was 'clearly European, and the architecture made it look like Switzerland or Austria,' but a second later he discovered a company or some kind of URL printed on the rear window of the car. Since it ended with '.at,' he guessed that the image was likely Austrian. Patterson is not familiar with the Austrian region, so he thought, 'There are some hills around, but they're not huge mountains. It's probably the eastern part of the country, a little way from the Alps, but still hilly.'
The results of this match are as follows. O3 achieved a high score of just 325m from the correct point. According to the chain of thought, he was able to guess the location quite close from the outline of the mountain, and even found out that the company's headquarters was in
When humans play GeoGuessr, it is considered cheating to perform searches during a round, but when they tried cropping the image given to o3 to remove information such as URLs and trying to identify the location from clues in the image without searching, it was reported that it was able to guess the location of the image as Dornbirn based on the shape of the mountains and the characteristics of the architecture.
Round 3
The image given in the third round was the following. From the yellow dashed line on the outside of the road and the white license plate, Patterson was 100% sure it was Ireland. Based on information such as the angle of the road and the land or island visible across the sea, he searched for a likely location on the map and marked it.
The results are as follows. o3 not only identified Ireland from the lines on the road surface, but also guessed that it was
In this round, o3 still used search, but the search terms were 'R477 coast road' and 'Galway Bay', which could only be found by identifying a very detailed location. From this point, Patterson says that o3 seems to have performed searches to confirm its guesses. Even when searches were prohibited, o3 still guessed almost the same location.
Round 4
The image given in the fourth round was below. Patterson immediately associated it with South America, and after noticing that the lines on the road looked Brazilian, he realized it was Colombia because all the license plates on the cars were yellow.
The cross on the back of the sign is also known as the 'Colombian Cross' and is also a characteristic of Colombia. Flat land like the one in the image is rare in Colombia, so I decided it must be east of the Andes or along the coast, and based on the feel of the trees and sand, I placed the marker along the Colombian coast.
The results are as follows: Just west of the Andes there was flat land that Patterson didn't know about, and the o3 had placed a marker slightly closer than Patterson.
Round 5
The image given in round 5 was below. Patterson first considered central or eastern Europe.
They then identified the roadside
Using the signs and other clues, he placed a marker in eastern Czech Republic, but immediately realized that the signs were for ' Michalovce' in eastern Slovakia.
O3 correctly guessed that the location was Slovakia based on signs, bus stops, etc., but the marker was placed in western Slovakia, close to the Czech Republic. So Patterson was slightly closer in a straight line, but the difference was small.
◆Match Results
Patterson's final score was 22,054.
o3's winning score was 23,179, resulting in their victory.
Also, o3 was not fooled at all by the fake EXIF information embedded by Patterson. When he embedded the EXIF information of a suburb of Bangkok, Thailand in the image he presented in the fourth round and asked about the EXIF information, o3 replied, 'I pulled the EXIF block from the JPEG, which included a GPS tag, but it was garbage for our purposes.' 'The coordinates show that you are in the suburbs of Bangkok, Thailand, and are clearly not near the Andean coffee zone as shown in the photo. So this file is a re-encoded Street View frame, with either fake or default metadata, or the camera that took the screenshot had old GPS information. In short, EXIF is more misleading than it helps.'
Patterson argues that while the o3 tends to spend too much time on unnecessary things and is easily distracted by unimportant things, it was still accurate in most of its guesses based on photo clues, wasn't fooled by fake EXIF information, and performed well without web searches.
Related Posts:
in Software, Web Service, Game, Posted by log1h_ik