Something I've been working on for a small while now is this little exploration/platform game engine that I've been making in Game Maker (because I'm a programming n00b). Except it's not just Average Joe's horrendous An Untitled Story/Knytt clone because rather than split the game into various screens and jump between them, it dynamically streams the world from the hard-drive creating one big large "load free" world :cool::cool::cool: I decided to make this because I'm awesome <more :cool: here>

www.kieranmillar.com/Streaming World Platform.rar

You can try it out if you want, but it's currently pretty small and has some awful level full of frustrating bits for testing purposes.

Arrow keys = move left & right
X = Jump
Down Arrow Key = activate checkpoints

Here's a quick rundown of everything the game has so far, mixed with an explanation of what the fudge you're looking at if you decide to play:

The black bits are ground (hopefully this is obvious). Green bits are jump-through platforms which you can jump up through but not fall through once standing on them. Blue bits are icy ground and the thin red bits are DEATH!
The yellow blobs give you an extra double jump when you collect them. Also, I'm aware that at the moment they come back if you leave the screen (did I mention I was lazy?). So after collecting two of them, you have two extra jumps in midair!
If you jump at a wall and push into it, you can wall jump! Ensure that you're moving into the wall you want to jump off of when pressing the jump button! Also, you can't wall jump off of icy walls.
The backgrounds might be hideous, but where you cross from one to the other is where the game goes about it's streaming business, so let me know what the slowdown is like. I've tried to do a lot to reduce it as much as possible.
The flat white things are checkpoints, and you need to PRESS the down arrow button when standing on them to set a checkpoint. They won't give you any feedback as to whether it worked (because I'm lazy), so trust me that it should work. (Note that holding down the down arrow button while running across them doesn't work).
The big large red arrows should be obvious what they do when you touch them, as should the red key and red keyhole blocks.

Finally, you have a jetpack! :cool: The big number in the top corner is your fuel, and you'll notice it drains slowly. The big "petrol pumps" will refuel you. Hold the jump button in mid-air to fire the boosters. In order to get the most boost, you should hold the jump button after pressing it down to jump. So if you wanted to jump, double jump, wall jump and then use your jetpack, you should keep the jump button held down after doing the wall jump and let go after the other jumps. The fuel is used up extremely quickly, so don't waste any!

Finally, if you do play the above example, the level essentially "ends" after that bit with the booster pads you need to fly through, after the exceptionally irritating bit where you jump through booster pads ON ICE. Trust me, you'll know it when you see it.

But there's more to making awesome games than an awesome engine, and that requires talent stupid ideas, so I thought after telling me how hideous my game engine is, you could all help make it much worse by giving me your horrible ideas of things to add to the game.

Bear in mind that there's a few limitations with the way my engine works. You'll notice that so far nothing else moves, and there's a very good reason for that! Because of the way objects are created/deleted on the fly, any object that requires interactions with other objects to work properly (except the player) is pretty much a no-no because I can't guarantee that the other object will be there. So an enemy couldn't "bounce" off of walls to turn around, for example.



Well gee this post turned out pretty long huh?

Bear in mind that there's a few limitations with the way my engine works. You'll notice that so far nothing else moves, and there's a very good reason for that! Because of the way objects are created/deleted on the fly, any object that requires interactions with other objects to work properly (except the player) is pretty much a no-no because I can't guarantee that the other object will be there. So an enemy couldn't "bounce" off of walls to turn around, for example.

That seems like a good thing to fix I'd say. :p

That seems like a good thing to fix I'd say. :p
Well, it could bounce off of walls, but if it moved out of the screen, there wouldn't necessarily be a wall there to stop it moving forever. Of course, the sensible thing to do in that situation would be to just delete the enemy object.

The major problem isn't that I musn't add any sort of interactions between other objects, but rather the problems that could arise with the way objects are created/deleted. Basically, every time I do the whole "streaming" business, I load and then put data about what objects should go where into arrays surrounding where I am. Then as I move around, I create/delete these objects as they move into/out of the screen. There's nothing to stop one of these objects from moving once I've created it, but then questions arise about how to go about deleting it. Currently, when I create an object it gets added into a separate array so that I can delete it depending on where it is in the "world grid", so if I deleted a moving object in the same way, it could be in the middle of the screen when I move where it was originally created into the "delete zone" and the object just pops out of existence when I'm looking at it, so this method isn't good.

HOWEVER, if I don't use this method, there isn't really a quick and reliable method I can think of that tells me not to spawn yet another one of those objects if I go back into the "create zone" while the previous one still exists. By running backwards and forwards in the right spot I could create a ridiculous amount of moving enemies, for example.

Of course, I could just empty the array position of that enemy when I create it so that it doesn't create more, at least until I offload the entire piece of that data and re-load it from the harddrive (which would involve moving pretty far away before this occured), which would be fine for enemies, but what about other moving bits like platforms? If I had to keep them existing until I deleted that entire section of memory, then they'd still exist way out of an area where an other object exists so they definitely couldn't be coded to bounce off of walls, but I could always hard-code them to only move within specific bounds. The only problem that arises then is that it still existing would eat up more time in my moving script when it was off-screen (because the player character doesn't actually move, instead the rest of the world moves AROUND the player, so I'd have to be moving these objects when not looking at them, and so if I use a lot of them, the game could be a bit slower).

Anyway that's more about the problems with other moving objects rather than interactions, but similar sort of problems would probably occur. Game Maker isn't very fast, so I need to be very careful about how I do things if I want to keep the game running at 60 FPS. Adding the odd object here and there isn't going to cause slow-downs, so with careful design maybe I could get moving platforms to be feasible as long as I limit how many I create in an area. "Goomba"-like enemies could work in a similar way too :p



Hopefully this post was fun for interested programming types.

It just seems odd design to me. What benefit is there from streaming the world directly instead of loading different screens if you're just gonna be so limited in what you can do in your world? :p

It just seems odd design to me. What benefit is there from streaming the world directly instead of loading different screens if you're just gonna be so limited in what you can do in your world? :p
Because I can, basically. I just wanted to see if I could do it.

Also, :cool:

ew. draw a new sprite for the player character. it burns!

also for the fuel pumps....

the game thus far is actually fairly good aside from that, though. the backgrounds hurt, but I understand the point, so it works. I'm getting a pretty smooth framerate throughout thus far. good job!

edit: that fake-out ice chute near the beginning is hilarious. good job.

edit: thanks to that boost-pad-on-ice area I have so many jumps I no longer need the jetpack! :cool:

Just to be clear, I haven't actually tried it, because it's not mac compatible. :P

uh....I don't think it's actually possible to beat the game. the last booster pad on that ice tunnel is at the top. you can't jump over it because it seems the character is the height of one full block of the level. even if you hover at the top and float forward slowly, you STILL drop a pixel and hit the pad.

edit: ok. I made it half across that pad without getting knocked back. that means it's possible. DAMN YOU, KIERAN! DAMN YOU!! I WILL beat this!

edit: OMFG! WHY IS THERE STILL MORE?! I BEAT THAT PART! WHY IS IT NOT OVER?! oh, lol. isn't that cute. you added fuel cans here. like we still need them at all.

ok. that was the end. and I can ONLY jump about 240 times now :cool:

I find it too easy to walk off the tiny platforms and fall to a red death. Perhaps a short tap of a left or right arrow should move him a much shorter distance, and he could accelerate to normal speed when you hold it for longer.

Apart from that, I like it!

Ideas:
- Variety in scenery so I don't get lost.
- Climbable walls.
- Tubes. What video game character wouldn't want to slide down tubes?
- Devices that shrink or super-size the character.
- Cloning device and parts where the characters have to help each other to get past.
- Device that turns gravity upside down for a certain amount of time
- A motor scooter.

Alright the game's kinda neat but explain this to me.

Something I've been working on for a small while now is this little exploration/platform game engine that I've been making in Game Maker (because I'm a programming n00b). Except it's not just Average Joe's horrendous An Untitled Story/Knytt clone because rather than split the game into various screens and jump between them, it dynamically streams the world from the hard-drive creating one big large "load free" world :cool::cool::cool: I decided to make this because I'm awesome <more :cool: here>Ok what.

I'll admit I don't know how Game Maker works, but this seems like a grossly inefficient way to make a scrolling level. If you're planning to do something more with it, like randomly generated levels or something, I might understand, but if you just want a scrolling level why not just load the whole level and make it scroll, instead of splitting it into chunks that you keep creating/destroying as you go and only being able to handle what's on the screen?

Ideas:
- Variety in scenery so I don't get lost.
- Climbable walls.
- Tubes. What video game character wouldn't want to slide down tubes?
- Devices that shrink or super-size the character.
- Cloning device and parts where the characters have to help each other to get past.
- Device that turns gravity upside down for a certain amount of time
- A motor scooter.minus those 2 I agree.

I also suggest:

-moving platforms (if you can manage a way to get stuff to move)
-trampoline platforms
-teleporters

I find it too easy to walk off the tiny platforms and fall to a red death. Perhaps a short tap of a left or right arrow should move him a much shorter distance, and he could accelerate to normal speed when you hold it for longer.

OK on the plane yesterday I implemented this. The acceleration is essentially halved for a short while, and when I quickly tap a directional key I only move about 1 pixel, so hopefully that should be good enough. Check if it's OK for you the next time I upload a new version.

Ideas:
- Variety in scenery so I don't get lost.
ew. draw a new sprite for the player character. it burns!

also for the fuel pumps....
Yeah, the graphics are only temporary for now. Maybe I'll make a bunch of different gradients or make the quick graphics not the same for now. Don't want to start slaving over beautiful graphics just yet.

- Climbable walls.

Ladders and the like is pretty obvious why didn't I think of it before?:-/

- Device that turns gravity upside down for a certain amount of time
- A motor scooter.
The anti-gravity is a nice idea, might take a small bit of effort to implement but is otherwise a neat idea.
Vehicles would also be a pretty cool way to mix things up. Maybe not a motor scooter (I already have boost pads) but some sort of flying machine perhaps?

Alright the game's kinda neat but explain this to me.

I'll admit I don't know how Game Maker works, but this seems like a grossly inefficient way to make a scrolling level. If you're planning to do something more with it, like randomly generated levels or something, I might understand, but if you just want a scrolling level why not just load the whole level and make it scroll, instead of splitting it into chunks that you keep creating/destroying as you go and only being able to handle what's on the screen?
While the biggest slowdown for Game Maker is having loads of objects existing at once, it has a neat feature called deactivation that lets you make large rooms whilst the game only "considers" what's in the surrounding area to you. I know Spelunky does this and so does IjI, and Iji's maps are pretty big so Game Maker can handle big rooms quickly. I've never actually attempted to see how quick deactivation is in a humongous room with thousands and thousands of objects, but I imagine it still remains pretty fast.
Really, the most major problem you face is memory usage. Even if objects are deactivated, Game Maker needs to store them in memory for when it re-activates them. OK, so this memory usage isn't really all that much, but once you're getting to hundreds of thousands of objects, storing their positions suddenly starts to eat up a considerable amount of memory. Imnagine storing the position, speed, sprite index, status, etc. etc. of thousands of objects and hopefully you'll see the problem.
Whether or not my method is worthwhile depends on just how large your project is. If the world is as large as an Iji level, then it's pretty dumb, but if it was the size of An Untitled Story's world, then you couldn't really store all that in memory throughout the game.

I also suggest:

-moving platforms (if you can manage a way to get stuff to move)
-trampoline platforms
-teleporters
I found a way to implement moving platforms which I added when on the plane as well. They aren't synched up yet (but that shouldn't be difficult), and getting my character to stop falling through vertical moving platforms was a nightmare, but they work now.
Bounce pads/trampolines are another "Doh why didn't I think of this" for me. They should be super simple to implement.
Teleporters are already technically implemented as you teleport when you die, so making an actual teleporter shouldn't be hard at all.

Oh and before anyone suggests it, I've already toyed with trying to make the world wrap around, but had so many problems that maybe I'll leave that for the sequel.

EDIT: If you're reading this now, then I updated the file, but mostly because I like to back up my work online. I removed the hideous gradients and added bounce pads and moving platforms. If you don't delete the old "Test World" folder then the gradients may still remain, but will be even more horrible because of a change I made to the way backgrounds are loaded.

Vehicles would also be a pretty cool way to mix things up. Maybe not a motor scooter (I already have boost pads) but some sort of flying machine perhaps?
I was thinking about something that could do jumps and ride loops, that sort of thing, but yeah, boost pads could do that.

Really, the most major problem you face is memory usage. Even if objects are deactivated, Game Maker needs to store them in memory for when it re-activates them. OK, so this memory usage isn't really all that much, but once you're getting to hundreds of thousands of objects, storing their positions suddenly starts to eat up a considerable amount of memory. Imnagine storing the position, speed, sprite index, status, etc. etc. of thousands of objects and hopefully you'll see the problem.
I'm not familiar with Game Maker, but I'm guessing one solution would be to convert groups of objects to files and remove them from the game, then bring them back in when you need to. I imagine it like a grid of points, with the points being a couple of screens apart. Each point would correspond to a file, and all the objects closest to that particular point would be saved in that point's file. When your character gets within a couple of screens of a point, you'd load that point's objects into the game, and if your character moves away again, you'd put them back to the file. It'd just mean that those objects wouldn't do anything when they're too far away.

I'm not familiar with Game Maker, but I'm guessing one solution would be to convert groups of objects to files and remove them from the game, then bring them back in when you need to. I imagine it like a grid of points, with the points being a couple of screens apart. Each point would correspond to a file, and all the objects closest to that particular point would be saved in that point's file. When your character gets within a couple of screens of a point, you'd load that point's objects into the game, and if your character moves away again, you'd put them back to the file. It'd just mean that those objects wouldn't do anything when they're too far away.
This is pretty much exactly what I do now. I have files which store object data which I load up when I get close enough. What I don't do though is re-save the files as I move away, because that would be really slow, and there's also nothing worth saving. If I want an item such as the red key to remain gone once I've collected it, I just immediately destroy it when it's created if a certain global variable is true (and the yellow blobs don't do that yet because they don't bother checking, when I do that they'll just be separate objects that check for different variables, rather than be the same object).

The bad news with that method is that I couldn't have too many objects with lots of things that need "saving" when you delete them, but I'm not sure what would really need saving anyway in most cases. Who cares if small enemies come back once you've moved 2 screens away?

What I don't do though is re-save the files as I move away, because that would be really slow, and there's also nothing worth saving.
It wouldn't be any slower than loading them in the first place would it? Although, if you decide you want to have a handful of objects which are worth saving, it wouldn't immediately be enough to cause a memory issue.

It wouldn't be any slower than loading them in the first place would it? Although, if you decide you want to have a handful of objects which are worth saving, it wouldn't immediately be enough to cause a memory issue.
OK I'm not too sure what you're getting at here. As my game is now, the vast vast majority of the objects are floor objects. There's no need to save these. The only objects that really need "saving" are powerups (have they been collected?) and... uh... maybe whether or not some simple enemies are dead (except bosses), but most games don't do this anyway to keep the enemies there to keep the areas interesting if you revist them, so this is unnecessary.

To give a practical example, if An Untitled Story used my engine, the only "objects" worth saving are the collectibles like the hearts/etc. That's like, what? 150 objects? Storing whether or not these are collected in memory is 300 bytes-ish? (Game Maker stores everything as a double, even variables that are only ever TRUE or FALSE) So saving objects back to the hard-drive is unnecessary. (Except when you save the game but we aren't talking about that, right? :p)

Also, saving to the hard-drive might take the same amount of time as loading them, but if I'm doing that on the fly, then I've just doubled the length of time it takes to move around the world (well OK not true, loading the backgrounds is what takes the most time). So there really is no reason for me to do it at all.

OK I'm not too sure what you're getting at here.

Never mind, have some ideas about other objects you might want to save (either in memory or in files):

- Unique enemies/friends who may be chased or run away to different areas, forcing you to search for them.
- Movable platforms, ramps, doors, switches and pushable or carriable objects which you don't want to reset all the time (perhaps because completing the game depends on their position).
- Objects which have a score or number associated with them. For example, a container you have to fill with a certain quantity, or a block you have to shoot 5 times to break.

While the biggest slowdown for Game Maker is having loads of objects existing at once, it has a neat feature called deactivation that lets you make large rooms whilst the game only "considers" what's in the surrounding area to you. I know Spelunky does this and so does IjI, and Iji's maps are pretty big so Game Maker can handle big rooms quickly. I've never actually attempted to see how quick deactivation is in a humongous room with thousands and thousands of objects, but I imagine it still remains pretty fast.
Really, the most major problem you face is memory usage. Even if objects are deactivated, Game Maker needs to store them in memory for when it re-activates them. OK, so this memory usage isn't really all that much, but once you're getting to hundreds of thousands of objects, storing their positions suddenly starts to eat up a considerable amount of memory. Imnagine storing the position, speed, sprite index, status, etc. etc. of thousands of objects and hopefully you'll see the problem.
Whether or not my method is worthwhile depends on just how large your project is. If the world is as large as an Iji level, then it's pretty dumb, but if it was the size of An Untitled Story's world, then you couldn't really store all that in memory throughout the game.Well if you plan to have the whole game be one huge "level", then yes splitting it into discrete sections is a must.

Don't get me wrong, optimizing is good and your concept seems clever. But never start optimizing too early, specially if it gets in the way of gameplay. You might be cutting corners when you're not even sure you need to, and stuff like enemies respawning when you scroll can feel a bit amateurish, not even Mario did that. Plus accessing the hard disk is a lot slower than memory, so you should try to keep it to a minimum. Even 100MB in used memory isn't that bad these days.

Hundreds or thousands of objects might seem like a lot, but most of them would just be in memory, not on screen, and even a million doubles should barely amount to 8MB (although why in the world would they decide to make every variable a double is beyond me). Plus most of them should just be fixed tiles with x/y, with only a few actual interactive objects with some data. Of course I write all this not actually knowing how Game Maker works internally.


Anyways I'm done being nerdy. :p But if you'd have trouble just implementing enemies because they might go through sections, consider just making the sections big enough so they'll never cross over (if they're the size of a background now, they look pretty small), or give them boundaries so they'll never go where you don't want them. See how far you can push it before you start cutting features.

Storing the state of everything interactive shouldn't be a problem since you'll mostly just need on/off and x/y type stuff, which you can just keep it all in memory as long as the game runs, and only dump it to harddisk on save points.

As for my terrible ideas, I already mentioned since you have a section-based engine, you could have the game be a bit more dynamic by having some sections getting swapped or changed around (when you're not looking at them, natch), either randomly or based on some game event. Make the world a bit more unpredictable. :p

Well if you plan to have the whole game be one huge "level", then yes splitting it into discrete sections is a must.

Don't get me wrong, optimizing is good and your concept seems clever. But never start optimizing too early, specially if it gets in the way of gameplay. You might be cutting corners when you're not even sure you need to, and stuff like enemies respawning when you scroll can feel a bit amateurish, not even Mario did that. Plus accessing the hard disk is a lot slower than memory, so you should try to keep it to a minimum. Even 100MB in used memory isn't that bad these days.
That's true. At the moment the memory needed for the gameplay is minute, and it's the graphics that eat it all up. Once I add in music and sounds, the memory would be a lot higher than it is now, but not so high that I couldn't splash out on lots of memory for the gameplay. The movement system at the moment is designed with low memory use and frequent (but small) access to the hard-drive to try and keep the game running as smooth as possible. This means that I have loads of memory spare for keeping other stuff in memory rather than saving to the hard-drive constantly, as my plan was to only save to the hard-drive on the actual "game save" rather than for small things on the fly.

Hundreds or thousands of objects might seem like a lot, but most of them would just be in memory, not on screen, and even a million doubles should barely amount to 8MB (although why in the world would they decide to make every variable a double is beyond me). Plus most of them should just be fixed tiles with x/y, with only a few actual interactive objects with some data. Of course I write all this not actually knowing how Game Maker works internally.
OK here's a basic run-down on how Game Maker works. A lot of people know that Game Maker is slow, and most assume that it's because it's an interpreted language, and while that helps, it's a tiny amount of slowdown compared to how the engine runs. Game Maker is designed for serious ease-of-use, so it does a considerable amount of checking and processing things which are totally unnecessary most of the time, and you can't turn these off. Here's a list of everything that every single instance of every object stores in memory (this includes every 16x16 "floor" block in my game). These might not all be doubles, I don't know, but I know that any variable you create yourself is:

The x and y position in the game world
The direction the object is facing
It's speed
Alternatively, it's horizontal and vertical speed if you prefer to handle things that way
Friction
Gravity and gravity direction
The unique ID of that instance
The "object index" (the type of object it is)
Which sprite it uses
The frame of the sprite it uses (this is also increases every frame)
How fast the sprite frame increases
How big you want the image
The collision mask for the object
Whether or not it's solid (if so, collisions are handled differently)
Whether or not it's visible
Whether or not the object will carry over between "rooms"
The drawing depth (the order objects are drawn in depends on this)
Finally, there's more variables determining the "path" of the object, which are all stored even if you haven't set one

But that's only memory usage. In addition, every "Active" object (you can deactivate objects so that they're not considered for processing) has the following things processed every single "tick" or "step" of the game, and is the real reason why Game Maker is slow:

Every object has three "step" events done at different times so you can set stuff to be done every tick. The game checks all 3 of these for every instance every step.
Every object needs to be drawn if it's visible, as well as have whatever's in the "draw" event be executed every step.
Every instance is checked to see if it needs to be moved and is done so if it does
The biggie - Every instance checks for collisions with every other one and then checks to see if it has a collision event to perform if a collision happens. I'm pretty sure it does this even if the object never moves.

I'm sure I read somewhere at TIGSource that someone advised that you should try to have no more than 50 objects active at any one time if you want a decent frame rate because of this. Crazy.

Anyways I'm done being nerdy. :p But if you'd have trouble just implementing enemies because they might go through sections, consider just making the sections big enough so they'll never cross over (if they're the size of a background now, they look pretty small), or give them boundaries so they'll never go where you don't want them. See how far you can push it before you start cutting features.
Yes, the sections are the size of a background. I can actually make them bigger just by altering a single constant in my code (I've made it flexible that way), but I can't get away with making the background images any bigger than they are now without some serious slow-down while they're loading. Of course, each section is currently split into 4 individual background images at the moment and loading them is staggered over 4 steps, so if a cell were to be made bigger, I'd need more background parts, but that wouldn't really be a problem because it would take longer to walk over to the edge of the section, so I could easily afford more steps to load them up (just manually chopping up my background images would be more of a pain in the butt). If I need to make sections bigger, then it's easily doable, and I can easily consider it if I need to, I just don't need to right now.

Storing the state of everything interactive shouldn't be a problem since you'll mostly just need on/off and x/y type stuff, which you can just keep it all in memory as long as the game runs, and only dump it to harddisk on save points.

As for my terrible ideas, I already mentioned since you have a section-based engine, you could have the game be a bit more dynamic by having some sections getting swapped or changed around (when you're not looking at them, natch), either randomly or based on some game event. Make the world a bit more unpredictable. :p
Hey, if I wanted to be even more stingy with my memory use I could even store things per section, so if I killed all the enemies in one section, I could set that section to "clear" and then the game won't load up the enemies from it the next time it loads some from there. Or alternatively, load a separate file the next time I go there to have the world change somewhat. But more realistically, because the entire game is "grid based" (each section is a 32x32 grid of objects), I could probably get away with storing a single value in a huge array to determine whether or not an object at that grid position should be loaded or not, or should have something else occur to it (so individual enemies could be flagged to no longer be created once killed for the remainder of the game, or destructible floors to remain destroyed or whatever). The feasibility of this depends on how big my world ends up being though. I'd also need to save this every time I save my game, but that's not a problem.
EDIT: Ok small calculation, a game with 100 x 100 sections with each grid square taking up one value in an array would be over 10 million values. I don't think the per-grid square idea is feasible in most cases.


Man this post is long. Sorry. You're helping me really think about how to handle things in my game and where to take things though, so thanks everyone!

OK on the plane yesterday I implemented this. The acceleration is essentially halved for a short while, and when I quickly tap a directional key I only move about 1 pixel, so hopefully that should be good enough. Check if it's OK for you the next time I upload a new version.
Nicely done, much better now.