This is awesome. Would definitely download a local higher-res version if that were an option, maybe using Google Street View data? Also, if this was a live MMO, that would be fun. Not that there's really a game aspect here other than the coins when driving a vehicle, but anything to give some life to the sim SF city would be nice.
Seemingly lower-hanging fruit options I'd like to see:
- show street names, maybe some landmarks or businesses, togglable
- enter address to teleport to
- enter address to get directions to, GPS style, to travel there manually
- get coordinates / current location as a URL to share
- zoom in on minimap, or expand minimap, so that teleporting can be more precise
There is so much scope with maps! All we need is a realtime-ish image model to tidy up the scene. (was looking into it during a recent side project - https://sf.isopolis.city/).
I lived in SF for nearly 20 years, though I mostly spend my time in New York now.
And genuinely this made me emotional. Was weird as hell walking around my back yard. Going to the spot in Dolo that I used to haunt in my 20s. My favorite tennis courts and hill bombs, walked the entire disc golf course, etc.
Happy to hear someone else say this. Similar story. I could never live in SF again but roaming around it was nostalgic in a way going back now just isn't
The free will of walking. I walked around the city using certain routes, and I can freely replicate those walks in this. I did walks I did when my kids were 3 years old. Google Maps isn't fluid enough to do this.
Honestly it’s a good question. obviously a big part is walking through it. But I also think it’s because it’s this weird, dreamlike version. Because it’s low res and a little broken, it’s like walking through a place you know really well in a dream. Little truths pop up, like a certain taqueria right where you expect it. But it’s almost a Silent Hill version of the town, so your mind fills in the blanks for you.
Maybe this is a common experience, but the first time I left SF for six months, I would have these sort of flashbulb memories of certain intersections and locations. Not even particular memories there. I think because I moved there right out of college and my late adolescence and all that it entails took place there, there are a hundred little corners where I’ve got a thousand more memories.
It being like a half-remembered scene particularly triggers the nostalgia. Also probably because I’m not there right now.
Same. I lived in SF for 15 years, my kids were toddlers in the Marina... this is so freaking awesome and really brings out the nostalgia! It matches my memories exactly.
Exact proportions are crazy. I had the reverse experience doing sim racing... do a track a bunch, finally visit it in person - and it looks the exact same as in your mind.
I’m a ‘serious’ sim racer and I’m gonna get the chance to drive a touring car at BTCC brands hatch in October. I’ve sim raced that layout a million times I’m very excited to experience this. Already had a weird enough feeling at Laguna Seca without driving it!
This made me super nostalgic as well. I've lived in NYC for 20 years, but spent a ton of time in SF early in my career (starting in 2008). I love NYC, but I've been sort of feeling like a change lately and this made me feel like subletting a place for a few months and getting back to the slower pace of SF.
I have dreamed about a pipeline that can take elevation data, building data, maps, and streetview imagery to process a city into all the components you would need to build a proper map for one of the GTA engines, including all the other objects like trees, fire hydrants, signs, light posts, traffic lights, etc.
The building textures could be recreated from the low-res streetview imagery using a pipeline to scale / relight them using image-to-image models. Building shapes could be rebuilt using generative models for 3d mesh components. Road & sidewalk quality (how clean, how faded lines are, etc) could be inferred from imagery as well to be applied to the map.
Super cool project, hoping it continues to evolve!
GTA games require a lot of effort outside of this, but the concept of mass-producing a GTA style open world game for any given environment is very compelling. Probably frightening to Rockstar themselves but such is the course of technological development.
Is this ripped using the retroplasma flyover-reverse-engineering code? The retroplasma repo is a bit behind now and doesn’t work with the updated way Apple serves map data and the heif texture format.
I’ve been playing with the same thing to make RC tracks in re-volt (or rather RVGL). This game though, running the game as a webpage is fantastic!
Looks to me like a weird combination; maybe the spat is trained off ripped Apple Maps data? The surface textures all look static like traditional photogrammetry (e.g. no specularity), but there are definite splat artifacts rendered.
Actually, ripping the data might not be required with this approach; instead the splat could be trained with a screen recording of normal Apple Maps usage.
'Vette was a racing game from the 90's (I think?) that covered San Francisco. "Released on 3 floppy disks", ha ha. Obviously without the rich dataset we have now. (Clicking on the following link ignores the trailing exclamation point in MacOS Safari for me.) https://en.wikipedia.org/wiki/Vette!
I did a voxel (ray casting) lunar lander that used actual NASA moon data (https://mooncraft2000.com).
Man, I just posted this same comment. 'Vette was the first thing I thought of. Ever since playing it way back then, I wanted a better SF open world game.
Ah nice demo! I flew around SF for way too long just enjoying the sights. On my M3 Air using Chrome it eventually just turns into blobs and my character get stuck, takes about twenty minutes. I popped open the DevTools and the game loads in at like 400mb but after five or so minutes I'm at 2GB of memory consumption. I'd guess that you need to have your agent do a couple of passes for memory leaks. Very fun otherwise!
Very amazing. And this is the worst it will ever be. We'll soon be exploring 4k versions of this and we can open every trash can in Tunis. Fascinating. Had a lot of fun (tedious tho) going out to Alcatraz and figuring out how to get in.
Similar concept, but more abstract in terms of art style. I'm pretty close to having the foundation I wanted before adding quests on top; telling a story is the ultimate goal.
Building on top of GIS data is super fun though. Especially since the barrier to entry is so low these days with LLMs. I encourage everyone to give it a shot if your local town/city publishes some data. No need to understand coordinate systems for example, which is nice.
For the game, there's a button to get in without an account so you can play around, but to get the fun mods you will need to play a bit and figure out the trash pick up/scrap at dump mechanic in the game.
(*might be a bit of lag; server needs more ram but on vacation rn!)
It took a little while because I was struggling with the scale a little bit at first, but I found my house. I appreciate that you tore 95 and 676 out! Keep it up - consistent art direction usually beats hyper-realism, for me, and I appreciate the bizarro world you've concocted.
Awesome! If you click the top left 'Mode' switch and go to Property, you can buy it :) you just have to pick up and scrap enough trash to get the $5k for it.
I do think 95 and 676 will be in the game more accurately eventually. But it will match the style of the rest of the game. It's a work in progress now but not good enough to ship yet. Love that you said bizarro, feels like i'm on the right track
This was so much fun! Some feedback - I wish the highways were more accurate, seems like a limitation of how elevation is handled on these maps. The navigation line was really hard to see since it was at camera level. It was hard to tell what direction i was going, the minimap and the large map seemed to disagree.
Hey, thanks. You're pretty much right about highways. I do have a separate working branch that has actual raised highways, but I'm still figuring out the kinks in making it look nice and joining the terrain correctly. Generating accurate entrance/exit ramps has proven to be my kryptonite but I'll get it eventually. I will give the minimap some love when I get back too-- thanks for checking it out
This is awesome and if it's legal I'd love to do similar things on geospatial projects I work on. However I don't think Apple will be OK with this use of their data. I know that Google closely limits its 3D tiles data through a specific API and Apple does not make a 3d API available, therefore this has to be against ToS.
From the ToS linked from OP site:
> 1.3 Restrictions. You will not and will not permit or authorize a third party to: (...) (xiv) decompile, decrypt, reverse engineer, attempt to derive the source code of or disassemble the Service or any portion thereof (except as and only to the extent this restriction is prohibited by applicable law or by licensing terms governing the use of any FOSS components that may be included with the Service)
I would love for people to prove me wrong that Apple will embrace this use of their geospatial data!
This would maybe be a pretty good opportunity to use some of that "upscaling" AI tech: given this weird results of spatial scanning, produce something plausible and much higher definition. There are some dangers of doing so, of course. People will believe details that show up in it because it's taken from real life, and AI loves adding fake details, but for a little game it seems like detail beats ground truth.
That was actually Watchdogs 2. Great game, basically GTA but you are a hacker in the Bay Area. The SF portion of the map was accurate down to every building.
If that second link is an in-game map, it is far from accurate to the real world. Unless it’s from a timeline in which Golden Gate Park is connected to the Presidio, and half of SF has fallen into the ocean.
Kinda funny game, though if you want a bit more fidelity, but less control, Microsoft Flight Simulator has a pretty good version of SF.
Though as for games that use SF (but not 1:1) Watch Dogs 2 is a pretty good proxy, though my favorite video game interpretation of San Francisco is from GTA San Andreas (San Fierro)
Very impressed with the Watch Dogs series of games for this purpose.
We recently used Watch Dogs Legion to prep for a trip to London to get more familiar with the city. Set the game on peaceful mode and just walked around and did virtual sight-seeing instead of playing missions. It's a surprisingly good recreation of the city, and we really appreciated getting some familiarity with the place before we visited.
I did a similar thing for the UK when prepping to drive around. I hooked my steering wheel up to Forza Horizon 4, and just drove slow cars on the wrong side of the road through Edinburgh, following as many of the road laws as I knew.
It wasn't perfect, but it did help a little. The streets are way narrower, and the traffic is much worse, but muscle memory to stay on the left side of the road was nice.
I do adore San Andreas but I think San Fierro is one of the weakest areas. My metric for how accurate they are is a sense of "oh I've been here before" and I really only get that in a couple of spots, as compared to Las Venturas (Las Vegas) and Los Santos (Los Angeles) where I could actually go to the cities in real life and feel like I knew where I was going because I'd played San Andreas enough. Compared to those two, San Fierro feels like a Backrooms-style dreamlike compression of the real thing, where a lot of stuff has been moved around. But parts of Haight-Ashbury, Pac Heights, Nob/Russian Hill, Polk Gulch, Ocean Beach and so on are still fairly distinguishable and correctly arranged.
Ha, Microsoft Flight Simulator is the most uncanniest valley thing I think I've ever played with; I'm from Tallahassee and it's weird to fly around a very high-fidelity, realistic, inaccurate version of a city I've lived in my entire life.
Have you thought of going to the SF DBI to get the indoor blueprints and digitizing them for even more experience? I wonder how good GPT Sol would do at scraping photos to apply to the blueprints one of some streaming geojson+bitmap or something.
This is incredible, I wish I could have this in my city.
Can’t help but wondering, how will this look like if we had AI try to “augment” these maps in real time (maybe using street view images?). I wonder if it would be playable in reasonable FPS, and how expensive it would be.
Beware, safari users. This locks up safari on my recent macbook pro and you cant close the window to clear it. Restarting safari doesnt help, the tab immediately freezes the app upon start. Holding shift while restarting safari doesnt help. Currently trying to figure out a way.
I'm sitting here watching the block bounce in a circle and I have no idea if the game is actually loading or not. Tried turning off PopUpOff. Tried turning off adblock.
as it turns out it just isn't loading (firefox on win 11, if that matters)
It always seemed a missed opportunity that Microsoft Flight Simulator doesn't have a UFO mode or something where you don't need to worry about the physics of keeping the plane in the air and could just stop and look at things.
Maybe you don't need real-ish physics for the glider here. Does the user really need to fall?
At some point someone will integrate actual interactive models of buildings and houses, based on Zillow pictures and whatnot, and then the entire thing will be interactive in a disturbing way
my first experience with San Francisco in a video game was through the City Escape stage in Sonic Adventure 2. and I didn't even know then that it was San Francisco (I had never been there when I played the game). only realized this when I visited the city many years later.
funny timing, i was exploring the same, some friends and me play alot of geogussr (a game where you put into streetview somewhere on the world, and have to find out where) lately.
I want to make it more video game like, cause the jumps of google maps make me motion sick.
I used google streetview as image source (cause you need to read signs, cars, see geometry) and colmap to reconstruct 3d chunks.
But it never really worked that well, jumps in time (cause the google car does not always continuously scan) and the image density is just too low, but was a fun side project.
This was loads of fun haha. I would have liked to have optionally clicked a button and started at my current physical location because the tile load time dominated how long it took to get there.
I was able to get it to stand directly on the roof of the structure I'm sitting at. (Which is kind of a glitch, it walked right over the structure as if it's a hill.)
When I noticed it getting choppy I realized I had filled most of my VRAM with an LLM, so I unloaded that and it got better.
I feel like I did that walk, in person (the tourist areas for sure). I've been to SF three times now, most of the time I'm walking everywhere. Even found a place that sells Brazilian coxinha
Looks like an atmospheric explorer kinda thing. Low-poly SF where you can just wander around and take in the city, pretty chill. I dunno, I really enjoyed it
When I was in SF the first time some spots seemed weirdly familiar to me. Only later that day I remembered that I probably know them from GTA San Andreas (but maybe also the Full House intro)
It is fun when you play a game based on a place, then visit the place. There have been a few places that I've been to that were levels of the Tony Hawk series. I enjoyed how the game designers were able to take what was otherwise pretty mundane, and tweak it into a fun map, while not just making it all up and losing the source location.
Back in the 90's, there was a game called "Midtown Madness" by Microsft. They simulated SF and London very well. This game now takes it to another level.
But what did available was a glider, so I guess my only form of faster transportation in the city is to glide to anywhere I want :)
Seemingly lower-hanging fruit options I'd like to see:
- show street names, maybe some landmarks or businesses, togglable
- enter address to teleport to
- enter address to get directions to, GPS style, to travel there manually
- get coordinates / current location as a URL to share
- zoom in on minimap, or expand minimap, so that teleporting can be more precise
- changeable avatar
- more dynamic weather / lighting options
So exciting!
And genuinely this made me emotional. Was weird as hell walking around my back yard. Going to the spot in Dolo that I used to haunt in my 20s. My favorite tennis courts and hill bombs, walked the entire disc golf course, etc.
Maybe this is a common experience, but the first time I left SF for six months, I would have these sort of flashbulb memories of certain intersections and locations. Not even particular memories there. I think because I moved there right out of college and my late adolescence and all that it entails took place there, there are a hundred little corners where I’ve got a thousand more memories.
It being like a half-remembered scene particularly triggers the nostalgia. Also probably because I’m not there right now.
Exact proportions are crazy. I had the reverse experience doing sim racing... do a track a bunch, finally visit it in person - and it looks the exact same as in your mind.
The building textures could be recreated from the low-res streetview imagery using a pipeline to scale / relight them using image-to-image models. Building shapes could be rebuilt using generative models for 3d mesh components. Road & sidewalk quality (how clean, how faded lines are, etc) could be inferred from imagery as well to be applied to the map.
Super cool project, hoping it continues to evolve!
Different approach than OP’s
I think it can do the whole world and just using webgpu
Dont recall name or more details, probably saw it on m3taverse account on ig
I’ve been playing with the same thing to make RC tracks in re-volt (or rather RVGL). This game though, running the game as a webpage is fantastic!
Actually, ripping the data might not be required with this approach; instead the splat could be trained with a screen recording of normal Apple Maps usage.
https://seattle64.benjasmin.chatgpt.site/
I did a voxel (ray casting) lunar lander that used actual NASA moon data (https://mooncraft2000.com).
https://www.mobygames.com/game/799/vette/screenshots/dos/
Curious, did you use a vehicle/glider, or did you take the heel-toe express all the way there?
Similar concept, but more abstract in terms of art style. I'm pretty close to having the foundation I wanted before adding quests on top; telling a story is the ultimate goal.
Building on top of GIS data is super fun though. Especially since the barrier to entry is so low these days with LLMs. I encourage everyone to give it a shot if your local town/city publishes some data. No need to understand coordinate systems for example, which is nice.
For the game, there's a button to get in without an account so you can play around, but to get the fun mods you will need to play a bit and figure out the trash pick up/scrap at dump mechanic in the game.
(*might be a bit of lag; server needs more ram but on vacation rn!)
I do think 95 and 676 will be in the game more accurately eventually. But it will match the style of the rest of the game. It's a work in progress now but not good enough to ship yet. Love that you said bizarro, feels like i'm on the right track
But so fun to play, bookmarked and will be back
From the ToS linked from OP site:
> 1.3 Restrictions. You will not and will not permit or authorize a third party to: (...) (xiv) decompile, decrypt, reverse engineer, attempt to derive the source code of or disassemble the Service or any portion thereof (except as and only to the extent this restriction is prohibited by applicable law or by licensing terms governing the use of any FOSS components that may be included with the Service)
I would love for people to prove me wrong that Apple will embrace this use of their geospatial data!
https://en.wikipedia.org/wiki/Watch_Dogs_2
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTbDzFt...
San Fierro was based on San Francisco of the 1990's. Lots of good memories playing GTA San Andreas and exploring 90's San Francisco.
Though as for games that use SF (but not 1:1) Watch Dogs 2 is a pretty good proxy, though my favorite video game interpretation of San Francisco is from GTA San Andreas (San Fierro)
We recently used Watch Dogs Legion to prep for a trip to London to get more familiar with the city. Set the game on peaceful mode and just walked around and did virtual sight-seeing instead of playing missions. It's a surprisingly good recreation of the city, and we really appreciated getting some familiarity with the place before we visited.
It wasn't perfect, but it did help a little. The streets are way narrower, and the traffic is much worse, but muscle memory to stay on the left side of the road was nice.
https://watchdogs.fandom.com/wiki/Category:Bay_Area
Can’t help but wondering, how will this look like if we had AI try to “augment” these maps in real time (maybe using street view images?). I wonder if it would be playable in reasonable FPS, and how expensive it would be.
Console shows CORS violations and WebGL errors.
Firefox on Windows. Recent ThinkPad provided by my employer.
I'm sitting here watching the block bounce in a circle and I have no idea if the game is actually loading or not. Tried turning off PopUpOff. Tried turning off adblock.
as it turns out it just isn't loading (firefox on win 11, if that matters)
Maybe you don't need real-ish physics for the glider here. Does the user really need to fall?
Oh yeah and multiplayer. Then ship it!
And stuff to do. Ship it!
I’ve long thought about a multiplayer MMO style GTA-like of San Francisco, though not literally 1:1 like this. Adds a whole nother thing.
Main thing is you have to bridge that Apple Maps -> game engine better though I must say driving off in the car was impressive
I managed to get stuck in a courtyard.
I want to make it more video game like, cause the jumps of google maps make me motion sick.
I used google streetview as image source (cause you need to read signs, cars, see geometry) and colmap to reconstruct 3d chunks.
But it never really worked that well, jumps in time (cause the google car does not always continuously scan) and the image density is just too low, but was a fun side project.
When I noticed it getting choppy I realized I had filled most of my VRAM with an LLM, so I unloaded that and it got better.
Pretty cool project.
I cannot turn the camera for the life of me. The camera is always facing north!
Edit: I feel dumb now, I needed to click the trackpad to rotate, somehow I expected a two finger drag to move the camera :facepalm:
https://gist.github.com/retroplasma/12e0e6e236aabd17c314cb53...
Thankfully the "reset to NY" button removes the chicken/eggs and lets me just fly through without any distractions.
I don't mind the lazy asset loading; I just can't go anywhere outside a couple blocks from the start position. M3 MBA.