Jump to content

Josh

Staff
  • Posts

    23,093
  • Joined

  • Last visited

Blog Entries posted by Josh

  1. Josh
    Version 2.32 is the biggest internal change in the engine in a long time. It's also the last time version 2.x will undergo this extensive of additions.
     
    LuaJit2 was implemented for fast Lua execution. This initially caused some BMX programs to not compile with the included modules.
     
    The rendering buffer storage scheme was modified somewhat in preparation for a new feature that is unannounced.
     
    Roads and saved to and loaded from SBX files, so they don't have to be generated at runtime. This caused some problems initially, too.
     
    A really efficient scene management system was implemented. This allows the engine to scale for very large games, especially when lots of small objects are off screen. It's a pretty advanced system, and there were initially some problems that had to be ironed out.
     
    A lot of this is stuff I want to have in version 3.0. I implemented it in 2.x because I figured it would be easier to first write in BlitzMax, and then translate to C++ later.
     
    This was probably the most difficult release we have done. One thing that magnifies those problems is the number of ways we have you can use the engine. Executing scripts from the editor, generating Lua glue code, etc. These aren't bad when everything is working, but it magnifies problems when they arise, because there is more that can go wrong. In version 3.0, we have a more uniform design centered around C++ and Lua. You can still use other languages, but I think the process will be a little simpler for us and for you. Having a stricter separation between "edit mode" and "game mode" in the editor will also help.
     
    Thanks to everyone who provided useful feedback this week, and happy coding.
  2. Josh
    I've been working a lot on the business side of Leadwerks lately, and although the results aren't yet visible, it's important to ensure our long-term goals are achieved. I wish I could say more right now.
     
    My main machine is back on Windows XP. I don't want to discuss the relative merits of operating systems, but for me Windows XP is more productive. Other people might not work the way I do, so they will have their own preferences, and no one is "wrong". That said, I am surprised with how simple everything feels. Instead of "experiencing" someone else's idea of what a computer should be like, I feel like I have things back under my control. I looked up a lot of registry tweaks, and am tuning XP to work just the way I want. Here are some of the modifications I made:

    Modified Explorer search to include all file types.
    "Computer" now opens with file tree on the left.
    Shortcuts don't display an arrow.
    Modified places bar shown on the left with file request dialogs.
    Disabled Autoplay on all drives.
    Disabled logon screen.
    Yeah, I know it is funny that a programmer is talking about Windows XP like it's 2001, but I like it.
     
    TweakUI is a neat little application I found handy that you might like, and it's offered on Microsoft's website, so it might be a little more reliable than some others.
     
    Another good aspect of my setup on XP is security. I love Sygate Personal Firewall. It prompts the user for every single connection that is attempted, and gives you the option to allow, deny, and save your setting to use again later. There's probably other Firewalls for Windows 7 that work just as well, but none of the ones I tried were as intuitive.
     
    I'm surprised with how few program I actually need. It's also interesting to see how they have changed over the years. Here's a partial list of installed apps:
     

    Firefox
    Digsby (I had a brief quarrel with this program, but I love it.)
    SmartFTP (inexpensive and fantastic)
    OpenOffice
    Microsoft Visual Studio
    BlitzMax
    BLIde Plus
    XNView
    Media Player Classic (I like Media Player 12 a bit, but it's not available for XP, and I always keep this installed anyway)
    WinAmp 2.95 with Blackdawn skin.
    Paint.NET
    Sygate Personal Firewall
    Avast Antivirus
    CDBurnerXP (very good free program!)
    Steam (of course)
    WinRar
    TGA thumbnail viewer
    DDS thumbnail viewer
    Windows Texture Viewer
    Adobe Reader
    So I have everything the way I want, in my own little obsessive-compulsive method of organizing my computer. I think today will shape up to be a very productive day, programming-wise.
     
    What common programs do you find useful? How have these changed over the years? Now that I have Digsby, I don't need Windows Messenger. I'm attempting to replace Microsoft Office with OpenOffice.org. What about you?
  3. Josh
    I like standard interfaces. I like the consistency of having all programs look the same. That's why web browsers bug me a lot sometimes; They each have their own unique look. It's disappointing to me how Microsoft has gotten away from this idea. Why oh why can't Windows have skins? But I digress.
     
    In terms of performance, my favorite browser would probably be Opera. In terms of appearance, my favorite is Internet Explorer 6. I know, web designers hate it for its quirks. I don't use it due to security issues. But I still like it because it is the only web browser out there with what I consider a simple standard interface. So I took matters into my own hands and found a skin that makes Firefox look like IE6. Then I replaced the program icon with the IE6 icon. I also edited some configuration settings to change the name of the program. I give you Firefox Explorer!:
     

     
    In all seriousness, the reason I am getting away from Microsoft Office is the ribbon interface. I've been using it since 2007, but I still prefer the Office 2003 interface. If they still had that as an option, I would be happily buying Office 2010. Maybe I am not representative of the average customer, but I have my likes and dislikes, and consistency is a big one for me.
  4. Josh
    Version 2.32 will feature some internal improvements that ensures your games run their best when conditions get demanding. I've totally rewritten the vegetation rendering routines with a highly specialized quad tree algorithm. There's a lot more to it than that, and I could write ten pages about the technique, but the bottom line is its fast and dynamic. Rendering with many vegetation layers is much faster, and since the visibility determination routine is so optimized, the engine can perform a separate visibility determination step for each stage in a directional light. The amount of shadow polygons that have to be rendered was less than half what version 2.31 required, in our test scene. Realistically, you can now render more trees with Leadwerks than Crysis.
     
    For entities, a scene octree has been implemented. This ensures that the engine only has to deal with objects in the near vicinity of the camera, even if there are tens of thousands of objects offscreen or far away. For scenes that have lots of little objects, you'll see a big performance gain, especially if you use the view distance feature. In version 2.32, each entity has a view distance setting which can be set to near, medium, far, or infinite. The engine stores lists of objects by view distance for each leaf in the octree, and skips object sets that are too far away, without having to iterate through each one. In the future, I would like to implement an update radius...this is how S.T.A.L.K.E.R. games are able to have so many AI characters walking around a map, and it's not too hard to add!
     
    Another improvement is the hierarchical rendering and picking algorithm. In version 2.31, every entity was stored in the scene graph. In version 2.32, only entities with no parent are stored. The top-most entity has a recursive bouncing box that encompasses all its children and sub-children, and is automatically updated once a frame when anything in the hierarchy changes. Let's say you have a truck. The truck consists of a chassis, four wheels, and a rotating turret. In version 2.31, all six sub-objects would be stored in the scene graph and iterated through for rendering. In version 2.32, only the top-most entity is stored in the octree. If the top-most entity is not visible, the entire truck is skipped. If the top-most entity is visible, its children are iterated through until they are all rendered.
     
    What about the case of animated characters, which might have 100 or more bones in their hierarchy? Wouldn't it be inefficient to have to iterate through the whole skeleton, especially when most of the time there won't even be anything that needs to get drawn? Yes, you are right. I got around this problem by adding a "renderable" setting to each entity. Meshes, coronas, emitters, etc., have the renderable setting set to True. Pivots, bodies, and other invisible entities have it set to False. Each entity also stores a recursive attribute that stores the number of children in its subhierarchy that are renderable. So when a bone is encountered in the rendering algorithm, if it doesn't have any renderable children in its sub-hierarchy, it will be skipped, and the engine won't have to iterate through all those bones for nothing. If the bone connects to a shoulder that connects to a forearm that connects to a hand that has a gun attached to it, the engine will iterate through those entities to get to the renderable gun mesh. It's the best of all worlds.
     
    If you only have a simple scene running, you probably won't see any speed improvements. However, these features will make the difference between your game running or not when you get further along in development, and will ensure you get the best possible performance. I would like to be working on some more high-level features that make the development experience nicer for the end user, but the most important thing for me to do is make sure your games are the best they can be. That's why I thought it was important to devote time to the described areas.
  5. Josh
    Day/night cycles are something I have thought about for a long time. There's several possible approaches to simulating these, but it wasn't until today that I decided which is best. Here are some options I considered:
     
    1. Shader-based skies with particle clouds.
    This is the method Crysis employs. A subsurface scattering shader creates the sky background. The mathematics are pretty complex, but the results for an empty blue sky look great. Particle clouds are used to place sparse clouds in the sky.
     
    Pros: Clear skies look great.
    Cons: Not very good for overcast skies, clouds look worse than a skybox, expensive to render.
     
    2. Animated skyboxes.
    The idea is to animate a looping sequence of skybox animations for a 24-hour cycle, and interpolate between the nearest two frames at all times.
     
    Pros: Potentially very high quality results, with a low cost of rendering.
    Cons: Long rendering times to generate skybox textures, no ability to tweak and adjust skies.
     
    The solution I finally arrived at is similar to the day/night cycles in STALKER: Call of Pripyat. A skybox with uniform lighting is used, so that it is not apparent from which direction the sunlight should be coming. The ambient light level, directional light color, fog, and skybox color are adjusted according to a set of colors for a 24-hour cycle. A large sun corona can be used to simulate the sun moving across the sky, and the directional light angle can be synced with the sun.
     
    Pros: Adjustable and dynamic, easy to add new skyboxes, low cost to render. Skyboxes can be changed to alter weather.
    Cons: Skyboxes with a distinct direction of lighting won't look good.
     
    Here are some of my results using this method:



    Of course, you are still free to implement any method you choose, but I am pretty satisfied with this approach for my own work, and I will make a scripted entity available to control these settings.
  6. Josh
    Until now, we haven't really had proper debugging info when a crash occurs during execution of a Lua script. Thanks to some previous work TylerH did with Lua, a conversation with him revealed how to easily add debugging info into the editor and script interpreter. (Incidentally, the Lua integration was Tyler's idea to begin with!) Here's a shot of the Script Editor catching an engine crash and displaying the script line the error occurs at. This is a crash that occurred in the engine, not a Lua compile error.

    This will make Lua a more viable alternative to other programming languages, and it will make it easier to use Lua as an extra enhancement to a C++ or other program. The integration of LuaJIT with the engine also speeds script execution up, a lot. Performance tests of the LuaJIT beta 2.0.0 reveal it to be up to 102 times faster than standard interpreted Lua. (Lumooja first told me this, and I thought he was exaggerating, but you can see for yourself.) And regular Lua was already faster than UnrealScript!
     
    In the future, I think we can look forward to having a nice display showing all the variables in your script program in real-time, and improved debugging tools.
  7. Josh
    Lots of things are happening around here right now!
     
    First up, we've got some free professionally made animations for you to use with our futuristic soldier character. You can download the files here.
     
    Michael Betke of Pure3D has been cranking out AAA-quality 3D models ready to use in Leadwerks Engine. The following sets are available:

    Birch Trees 1.x (€22,00)
    Pine Trees 1.x (€22,00)
    Ground Plants 1.x (€19,00)
    Each set features new unreleased content. Buy a couple today and help support low-cost AAA media for independent developers.
     
    I've been spending a little time with C++, and it's not that bad. Mika Heinonen was kind enough to investigate a method of integrating Lua with C++. Here are the findings he reported:
     
     
    At this point, we have a simple C++ project with some math classes the engine uses, and we're making sure it compiles for all the various platforms I want to support.
     
    Finally, we're getting ready for the release of version 2.32. There are some physics features that need to be finished to support huge amounts of trees and rocks, and that information has been sent to the author of Newton Game Dynamics. Numerous bugs in the tracker have been resolved recently, and more fixes are on the way for small issues.
     
    That's all for now!
  8. Josh
    If you prefer the classic start menu like I do, there are several solutions available to get the classic start menu in Windows 7. My favorite is Classic Shell.
     
    Classic Start Menu is a clone of the original start menu, which you can find in all versions of Windows from 95 to Vista. It has a variety of advanced features:

    Drag and drop to let you organize your applications
    Options to show Favorites, expand Control Panel, etc
    Shows recently used documents. The number of documents to display is customizable
    Translated in 35 languages, including Right-to-left support for Arabic and Hebrew
    Does not disable the original start menu in Windows. You can access it by Shift+Click on the start button
    Right-click on an item in the menu to delete, rename, sort, or perform other tasks
    Available for 32 and 64-bit operating systems
    Has support for skins, including additional 3rd party skins
    Fully customizable in both looks and functionality
    Support for Microsoft’s Active Accessibility
    And last but not least – it's FREE!

     

  9. Josh
    Pure3D has launched their realtime division for producing high-quality real-time assets ready to use in Leadwerks Engine. The first vegetation model set Groundplants 1.x is now available. It features 20 digital vegetation and foliage models. More file formats and engines will follow. This pack is available at a special introductory price of only €19. You can browse the contents of the pack in this pdf file.
  10. Josh
    I got my hands on the GDC showcase scene by Michael Betke, and it's much more beautiful than I thought. I used this scene to rewrite and test the vegetation rendering code. Collision isn't working yet, but it's built into the routine. The results speak for themselves.
     
    Fortunately, Pure3D is soon offering a set of game-ready vegetation models, so you can create gorgeous landscapes like this.
  11. Josh
    I got selection and rendering order working correctly, which is really the problem I wanted to solve with this. Here's a screenshot of the work in progress:
     



  12. Josh
    I skipped the GDC for a couple of years, because I just wanted to focus on developing our technology. This was my first year back since our engine came to be. I was surprised how many people were familiar with our engine, and I got to talk to a few users (who probably aren't active on the forum). I also got to chat briefly with Tom Spilman, David Helgason, and Mike Wuetherick. It's really fun to see what everyone is up to, and how far they have come since a few years ago.
     
    We met new friends including the FaceWare team, the PureLight guys, and the GameDev.net team (they are launching a new site soon, so stay tuned!). I got to talk at length to some nVidia employees, and we had a very interesting discussion about the future of mobile graphics...very interesting.
     
    Oh, did I mention OpenGL 4.0 was announced while we were there? I am very excited about this new technology, and really happy with how Khronos has come through for us.
     
    We also got to talk to a few potential investors. There is a well defined plan for the future of Leadwerks technology, but I can't make that information public quite yet.
     
    If you've never been to the GDC, I highly recommend you attend this event at least once. We've uploaded a ton of videos on our YouTube channel to give you a taste of what it's like. I'm looking forward to GDC 2011, especially if we have a booth of our own. If you're looking for publicity for your game, we will be looking for projects to show the potential of the engine, and there's a good chance we can get your game seen.
  13. Josh
    Dave Lee has joined the Leadwerks team. His job is to create a AAA game environment in Leadwerks Engine. Check out his work here.
     
    (Influences of Crysis and S.T.A.L.K.E.R., how could I not hire him? )
     
    Simon Benge, the lead artist from FPS Creator X, is also completing some animations for us.
  14. Josh
    I dabbled in a lot of little things recently, like GUI code and networking, just to make sure things would work the way I want in the long run. The lighting optimizations weren't planned to come out yet, but were moved to the front due to one user who is finishing up a commercial game. I've also been helping Pure3D a little bit, but I can't say anything about that right now. Of course, our website is still in progress, and that will hopefully come together by the end of the month.
     
    We'll have some cool stuff for you guys at the GDC. If you happen to be going, speak up now!
     
    There are two big things I want to complete in the next two months. They are:
     

    Art pipeline enhancements. The plan is to have a model viewer and editor integrated into the main editor. There's lots of little things like normals, binormal/tangents, physics, and the mesh hierarchy that can be difficult to export properly. This tool will give the user more control over their assets, and make it easier to import content from a variety of sources.
     
    Vegetation system improvements. Some parts of the vegetation rendering routines will be rewritten for better speed with large numbers of layers. I'll also add collision, while I am in there.
  15. Josh
    I've never considered myself a fan of writing GUI code, but now that I have a small job I need to get it done for, I'm actually having a lot of fun with it. I've been tinkering around with Lua GUI code for two days, and I came up with a really neat system after lots of trial and error.
     
    Controls

    A "control" is the name for a button, label, or other widget. A control is a Lua table.
    The programmer creates the controls they want and then inputs mouse coordinates and the left mouse button state in the Update() function.
    There are only three real events that occur; mouse movement, mouse button down, and mouse button up.
    When the mouse is pressed, the system has a fast hierarchical search to find the selected control, and this control is made active.
    Since the programmer supplies the mouse coordinates and button state, the GUI could be displayed on a 3D surface and interacted with in the game, like in Doom 3.

     
    Frames

    Frames are a lua table with a drawing method. The simplest frame just draws a rectangle on the screen. An image frame draws an image within the specified coordinates. A more sophisticated frame uses nine images to draw image-based edges around a rectangle of any size.
    Controls use frames to draw different parts of the control, instead of using a lot of hard-coded GUI drawing or images.
    You can replace a control's frame easily in Lua:
    button.frame=CreateImageFrame(image)


     
    Methods
    There are some predefined methods you can add to a control. Right now I have the following:

    CursorDown
    CursorUp
    CursorMove
    Activate
    Deactivate
    HideUpdate
    ShowUpdate

     
    Lua makes it really easy to expand a basic control to look or act differently. You can just overwrite a function, and you have your new control. For example:

    function CreateMyButton(text,x,y,width,height,group,style) local button=CreateButton(text,x,y,width,height,group,style) function button:CursorMove() --Your code here end function button:Draw() --Your own drawing code goes here end return button end
  16. Josh
    Here are some of my thoughts on how a future engine might look. I am not devoting any time to this, and it would not come to exist for several years, but it's fun to think about technology and design:
     
    I've been fascinated with the game "Fuel". This game has an enormous playable area that is streamed from the hard drive. You can drive for hours and barely cover a fraction of the map. For some reason the game has been given bad reviews. I played Motorstorm on the PS3, and I think Fuel is a lot better than that game, not even considering the impressive technical features. Anyways, I love the large open world. Fully dynamic lighting was one holy grail of game development, and I think a continuous open world is another.
     
    I would start this with a concept of "regions". Each region is a square on the grid, perhaps 1024x1024 units. The region has a terrain heightmap, terrain normal map, and an SBX file for all entities in the region. As you move around the world, regions are loaded from the hard drive in a square of playable area around you. Maybe there are 16 regions active at any given time. This number should be adjustable. Only physics and scripts of the entities within this active region are updated. When a region goes out of the playable area, it is saved to a file, and this new file is loaded if that region re-enters the playable area. So you aren't going to have enemies fighting and moving around when they are a million miles away. The engine just updates the regions nearby, because it would be simply impossible to update millions of entities. The AI in STALKER actually uses a technique similar to this, and I think they originally were planning on a huge continuous world but weren't able to pull it off. Each region can be culled on a separate thread, so this design lends itself to multithreading.
     
    So far what I have described is a big challenge, but it seems feasible. The real nightmare begins when we talk about networking. If your friend walks ten miles away and interacts with a person who's attitude towards you changes based on the interaction, well the server needs to be running two simulations. One simulation is for you and everything around you. The other simulation is for your friend and everything around them. What your friend does will change the world outside your playable area, so that when you enter that region it will be different than when you left. Now consider physics. Due to the massive distances, there would need to be two separate physics simulations running. Then they would need to be merged together when you approached each other. Wow, no wonder no one has done this yet! Fuel has a multiplayer mode, but it's just a racing game. Your actions don't have any permanent consequences on the world around you. An FPS, adventure, or RPG game will be quite different. I'm not even sure this is technically possible, without severe restrictions in flexibility.
     
    Fortunately, real-time rendering is not going to change much in the next ten years. We'll be able to do more as hardware advances, but I think we'll still be using the same basic techniques for a long time. That means that every feature and technique we add now will still be useful for a long time, even in a new engine.
  17. Josh
    Their prices are more than 250% the price of a comparable PC. Even their "most expensive" laptop is using a budget GPU! So for less than half the price, you can actually get a laptop with a much better GPU!
     
    $999:
    http://www.newegg.co...=laptop%209800m
     
    $2499:
    http://store.apple.c...mco=MTM3NDcyOTc
     
    Just for fun, I checked all the most expensive options. The grand total was $7305.35. Who spends $7000 on a laptop with a budget GPU???
     
    I can't believe how dense their management is. It's as if they don't want to succeed.
  18. Josh
    I got my idea working with spot lights! You can simply use a shadow mode of 2 to indicate an object or light should be considered "static". A light that has the static mode set will use two shadow maps; one for static objects and one for dynamic objects. In the image below, the walls and room are static, and the oildrum has the regular dynamic shadow mode set. As you can see, redrawing the shadow only requires 650 triangles. Without this feature, the whole room and everything in it would have to be redrawn any time something moved!
     



    Best of all, if the light moves or if any static object within the light's volume moves, both the static and dynamic shadowmaps are redrawn. The process is seamless for the end user. Dynamic lighting is usually dynamic, and baked lighting is usually static. I'm not sure what this qualifies as, but one thing is for sure...it's optimal!
     
    This will yield a huge performance increase for scenes with lots of point and spot lights, as long as those lights remain in a fixed position. You should avoid having lots of moving point and spot lights.
  19. Josh
    In Leadwerks Engine 2.3, two rendering paths are used for point lights. One uses a depth cubemap (for shader model 4 hardware). The SM3 fallback uses an unwrapped cubemap on a flat texture. On SM4 hardware, a depth cubemap was used because it allowed a point light to be rendered in one pass. However, when implementing my double-buffered technique for point lights, some problems arose. As a result, I changed the renderer to draw point lights as a series of six pyramidal volumes. This is how S.T.A.L.K.E.R. drew point lights, at least in Clear Sky. There was also a problem with NVidia cards not being able to read from a color cubemap unless it had mipmaps. Think for a moment about that, and you can probably guess where I am going with this...
     
    Now SM3 and 4 cards will use the same render routine for rendering point lights. SM3 hardware may be a bit faster, because the unwrapped flat cubemap texture coordinates took a lot of logic to figure out, but I don't think it will make any significant difference either way.
     
    I talked earlier about separating geometry into "static" and "dynamic" objects, and how the engine gained massive savings by only redrawing dynamic objects, unless otherwise needed. I took the concept a bit further, and decided to allow a light to cast shadows from static or dynamic geometry, or both. Consider the image below. The fan is in motion, but only about 1000 shadow polys have to be drawn to update the shadow:
     



    The large spot light projected through the fan blades is the dominant light, but the point light above the fan also casts shadows. If you look carefully, you may notice the oildrum in the foreground is not casting a shadow from the point light. The point light is set to only cast shadows from static geometry, so it never has to be redrawn (unless you move a static object in the editor). Because there are more dominant lights in the area, the viewer probably won't notice this technical error. This allows you place extra point or spot lights throughout a scene to create more detailed static lighting. The shadows will never be redrawn, and you can use them together with fully dynamic lights to create more lighting detail, without the cost of rendering more shadows.
  20. Josh
    I got the volume intersection tests working, and it won't be hard to extend this to make trigger and fluid volumes. The image below shows a point light divided up into six volumes. Each pyramidal volume represents the camera frustum of one face of the shadow cubemap. In version 2.3 and earlier, whenever an object moves within the spherical volume of the point light, the shadow is considered invalidated, and all six faces of the shadowmap are redrawn. This is why point lights are so expensive to render compared to spot lights, which only require a single draw when the shadow is refreshed. The shadow updating was originally designed this way because it was thought that a geometry shader on shader model 4 hardware would allow rendering all six faces in one pass and provide a performance improvement. However, geometry shaders didn't turn out to be a good solution.
     



    In version 2.31 the engine will only redraw those faces that have been invalidated. Volume tests are used to determine which volumes an object intersects. The oil drum in the image would probably only invalidate one volume of the shadow if it moves. Of course if the light itself moves, all volumes are invalidated, but you already know a moving point light can be expensive to render.
     
    There's another technique I want to implement, which I am calling double-buffered shadow maps. This involves storing two copies of a shadowmap, one for static geometry and the other for dynamic geometry. In the example below, the static room and equipment would be rendered to the static shadow map, and the oildrum and any other moving objects would be rendered to the dynamic shadow map. Whenever the dynamic shadowmap is redrawn, the contents of the static shadow map are first copied to it. If any static object moves, both the static and then the dynamic shadow map would be updated. This would make it so that instead of re-rendering all the objects around the point light, the engine would only have to copy the static shadowmap contents, and then render the dynamic objects on top of that. This will yield a huge boost in performance in scenes with lots of point lights.
     
    I've had these ideas for a long time, but the reason I am devoting time to them right now is because a game is coming out relatively soon that uses Leadwerks Engine, and I want it to be the best it can be. You know who you are.
×
×
  • Create New...