Multithreading GLFW?

theagentd wrote on Wednesday, December 16, 2015:

Hello. I’ve been thinking about a solution to a few problems.

  1. Performance issues when using multiple controllers, where polling and checking if they’re still connected is quite slow. This will most likely be improved in the future as it is an open issue on GitHub, but with how OpenGL is single-threaded all milliseconds saved will lead to a performance increase, and it’s safe to assume that all computers will have an extra core that a high-priority input/event polling thread can utilize.
  2. Unifying callbacks and guaranteeing that they’re executed on a specific thread. As far as I know, there is a risk that some callbacks are called on other threads. This would simply be a way of synchronizing and buffering all callbacks into a single callback list and allowing the main thread to handle them whenever it’s ready for them.
  3. Improve responsitivity of the game window(s). GLFW windows aren’t updated (dragged, resized, maximized, top bar updated, tabbing out of fullscreen, etc) until glfwPollEvents() is called (this might differ depending on windowing system???). This is problematic when our game is loading, and it’s difficult to ensure that the game’s loading screen is updating at a fast enough rate for the window to remain responsive. Joystick input may also be missed as it requires polling at a high rate (another open issue that is currently being worked on).

The idea is to call glfwInit() on a separate thread (let’s call it “the GLFW thread”) and do all non-thread-safe calls on this thread. For example, when the game thread wants to create a window, it passes a task to the GLFW thread and awaits its completion. The job of the GLFW thread is to do joystick and input polling, and call glfwPollEvents() at a high rate to ensure that all windows remain responsive even if the rendering freezes up during load screens. All callbacks that are triggered are buffered in a synchronized queue which the game thread empties when it is ready.

The GLFW thread’s loop looks something like this:

    processTasksFromGameThread(); //commands that have to be run on the GLFW thread.
    pollJoysticksAndGenerateEvents(); //until joystick callbacks are implemented.

This solves all three problems above. With multi-core CPUs the sometimes slow GLFW joystick/event handling can be offloaded to a separate core to make them pretty much free (although they might collide with the game’s worker threads sometimes). Since it gathers all the callbacks into a queue some threading issues are resolved. Finally, all windows will remain responsive even if the rendering/game logic freezes up for some reason, which helps make the game look more “professional” and removes some of the burden of making sure that the game’s load screen always updates regularly, something many games don’t do correctly. Rendering can continue as usual since I’ll just bind the OpenGL context to the game’s thread like before.

Does this seem like a sane thing to do? Are there any problems with doing this?

Some notes on your points first:

  1. My own experience is that controller polling & checking isn’t slow, and the only github issue I can find is related to trying to poll more frequently when the application is slow by using threading. Since OpenGL calls can block, this is a real issue, though a first order solution is to keep frame rates high and potentially poll more than once per frame.
  2. All callbacks occur on the main thread, and all event functions and controller polling functions need to be called from the main thread (the poll events function calls the callbacks, so they happen on that thread and the OS only posts events to the thread where the window was created so polling needs to happen on that thread).
  3. It’s always a good idea to either time slice or multithread long processes such as loading. Many games use a secondary thread for loading for this reason, but you don’t need to run the OpenGL creation calls on the same thread as data load.

However your ideas seem reasonable, though another approach might be to keep all window and event handling on the main thread and move the OpenGL calls to another thread using the context handling functions.

Currently my own approach is to do all window and OpenGL calls on one thread, keep the frame rate high and do any slower processing using my own tasking API enkiTS - there’s a basic example with glfw called enkiTSMicroprofileExample. I’m considering adding thread affinity for tasks so it’s easier to offload OpenGL calls to one thread.

I don’t think glfw needs any changes to make any of this possible.

theagentd wrote on Thursday, December 17, 2015:

Thanks for your answer.

I think that callbacks may be called from other threads in some rare cases:

Do not assume that callbacks will only be called through either of the above functions. While it is necessary to process events in the event queue, some window systems will send some events directly to the application, which in turn causes callbacks to be called outside of regular event processing.

Although we have made sure our loading is fairly multi-threaded, I’ve encountered lots of situations where the main thread is blocked due to driver weirdness and other annoying stuff I can’t control. Ensuring high framerates all the time and that nothing ever blocks the main thread is something that I haven’t been able to guarantee 100% due factors beyond our control is what lead me to consider this.

In our case it’s easier to place GLFW on its own thread than to place the OpenGL context on a different thread. In addition, the goal is to make sure that GLFW can work uninterrupted regardless of game logic or rendering, so placing GLFW on a lightweight thread that just spins on glfwPollEvents() seems like the best solution IMO.

That paragraph regarding callbacks means that some OS callbacks occur outside of the glfwPollEvents / glfwWaitEvents function calls, but they still occur on the same thread. Note that the glfw docs specify the ‘main thread’ but most OSs post events to the same thread as the window was created, so keeping all your glfw init, window and event calls on one thread should work (EDIT: Note see Camilla’s response below this will not work across all platforms)- I’ve not tried this and it’s not an area of code I’ve worked on in glfw, and so keeping these glfw calls to the main thread as specified by the docs may be safest approach.

Cocoa has an event queue per thread that can only be accessed from that thread, but the main thread event queue is the one that receives window and input events, and most window and view operations may only be performed on the main thread. Win32 has one event queue per thread and windows are tied at creation to the queue of the thread that created them, but most window operations may be performed on any thread. X11 has a single event queue and both event processing and window operations may be performed on any thread. The limitation imposed by the GLFW documentation is to ensure that programs are portable. This limitation is not enforced, i.e. the library does not try to prevent you from shooting yourself in the foot.

If loading or rendering blocks your program, put it in another thread. Everything you need to render and swap buffers is thread-safe on all platforms for this reason.

Thanks Camilla - that clarifies things a lot!

theagentd wrote on Thursday, December 17, 2015:

Thanks a lot for all the responses, but I’m still a tiny bit confused. Since the documentation mentions that callbacks may be fired off outside of glfwPollEvents() and glfwWaitEvents() I assumed that this meant that some callbacks could possibly be called from a different thread. Does this simply mean that some callbacks may be fired when functions other than glfwPoll/WaitEvents() are called, for example glfwSetWindowPos() immediately firing off a WindowPosCallback from within that function call (= on the same thread)?

Yes - the callbacks occur on the same thread but can occur outside of the glfwPoll/WaitEvents() function calls (as I wrote a few messages up), for example as you mention when you call glfwSetWindowPos().

Because of the Cocoa restraint Camilla mentions, you should follow the documentation on which thread to call functions from - so your window, controller and event handling should be on the main thread.

theagentd wrote on Thursday, December 17, 2015:

Thanks, that was what I was misunderstanding. In that case the event buffering isn’t necessary. Thanks a lot!

Thanks, this thread has just saved me from a possibly ever lasting state of confusion and compulsive googling. More specifically, the glfwSetWindowPos()/WindowPosCallback() example did it for me. Before that example, I really had a hard time understanding how callbacks could occur on the same thread, while outside of glfwPoll/WaitEvents(). I am still curious to understand how that happens under the hood (how does glfwSetWindowPos() invoke the callback without willing it?), but for my own programming, I can just accept it. Since it is not trivial and some people are getting confused, maybe the example could be added to the documentation.

Apart from that, as a GLFW beginner, I am really impressed by Camilla’s work. Did she start the project from scratch or did she take over the maintenance?