Threading is Hurting My Brain

[Thorin]

So I'm working on an application that reads from multiple serial ports.  The easiest way to do that, of course, is to call a new worker thread to handle the Data Received event each time data comes in on a serial port.  Unfortunately, the data needs to be recorded into a log, and if it's unique it needs to be recorded into a database.

Well, I've heard (and read) that Threading Is Difficult^tm.  So I thought I'd go through a tutorial or two to refresh my memory on how threading works in .NET, and more specifically what the gotchas are.  I managed to find this very thorough discussion on threading in .NET, complete with C# examples (even .NET 4.0):

http://www.albahari.com/threading/

However, my brain now hurts.

[Lazybones]

Hello blocking and locking.... However the performance gains on modern hardware are amazing...

Didn't get to far with it when I was still developing but I did make one or two multi-threaded applications... Debugging has to be the hardest part.

[Tom]

One alternative is using an "IO Multiplexer" which is just fancy words for "select". I'm not sure how .NET or Windows handles that though. If its possible to ask the OS to wait on several devices at once, thats an easy way out.

My game server queues up all its sockets, and select's on them at the same time.

But yes, threading can be quite hard. Especially if you need high performance and scalability. If you don't just use a nice global mutex around the common data structures and call it a day.

[Thorin]

In the case of this particular application, I'm not multi-threading for performance gains, I'm doing it to be able to respond to multiple hardware signals at the same time.

In the end, I'm taking the easy way out: I catch a hardware signal on a different thread, then marshal it back to the main thread where all the processing happens.  This ensures sequential updates of member variables without having to think about locking.  Yeah, it means no performance gains, but really, this app doesn't do much besides catch hardware signals and report them in a UI.

I wouldn't mind some day working on an app that requires threading for performance, that is properly planned out, and that wasn't due before they asked me to work on it. <sigh>

[Tom]

In the case of this particular application, I'm not multi-threading for performance gains, I'm doing it to be able to respond to multiple hardware signals at the same time.

In the end, I'm taking the easy way out: I catch a hardware signal on a different thread, then marshal it back to the main thread where all the processing happens.  This ensures sequential updates of member variables without having to think about locking.  Yeah, it means no performance gains, but really, this app doesn't do much besides catch hardware signals and report them in a UI.

I wouldn't mind some day working on an app that requires threading for performance, that is properly planned out, and that wasn't due before they asked me to work on it. <sigh>

My first suggestion is pretty much identical, except you don't have to create any threads. Just sit and "wait" on all the devices, and when one or more have data, you'll be woken up to handle it all.

[Thorin]

If I stop my main thread to wait for hardware signals, a la your suggested select(), my UI stops responding.  For those in the .NET world, select() is essentially the same as Thread.WaitOne().

So given that select() blocks the thread it's called on for a given amount of time, how do you select() on more than one socket at the same time from only one thread?

I'm willing to bet that you're actually creating a new thread for the socket, then calling select() on that thread while waiting for it to answer.  Especially as this is the standard way of handling multiple sockets in an application.

[Tom]

Nah. If .NET's gui stuff is half as good as Qt you can register as many "Devices" with the main loop as you like and it'll do the waiting for you. At least my game server works that way. I just get signals when theres a new connection or theres data available.

[Tom]

Also, select really isn't like a Thread.waitOne, select can wait on multiple handles, up to 1024 or so on linux, and if the timeout is 0, it just returns asap rather than sleeps. I can't tell if Thread.waitOne is just WaitHandle.waitOne...

So yeah, you can use select in non blocking mode, or fire off a /single/ thread to notify the main thread that things are happening. (not one per device)

[Lazybones]

Rule 1 of threading / ui is that you do your "work" in a different thread than the ui.
Two threads main/ui and a single work thread would be easier to sync than a main/ui plus a thread for every interface.

[Thorin]

I'm breaking Rule 1.  Although none of the work takes more than a few milliseconds in my app.  The only reason there's multiple threads is because whenever an event happens, the event handler is started on its own thread.

From what I saw of select(), if you set the timeout to 0 then when it returns it is no longer waiting for a signal from the hardware.  It might be that what happens is a new thread is spawned that sits and waits for response from the device.

[Mr. Analog]

You're threading hardware calls? Madness!

There are a few things you NEVER want to do in a multi-threaded environment:
1. Database calls
2. File IO
3. Anything that should be transactional

The app I work on is a total mishmash of someone not understanding how threading works in the first place and then trying to solve their problems by nesting thread pools inside thread pools. And the client wants us to "fix" their little transaction "problem".

Before I got my hands on the code there was not a SINGLE LINE in over 500,000 lines of code that EVER prevented thread locking, ANYWHERE.

Anyway, it's a sore spot and I learnt a lot about how to totally abuse multi-threading beyond all belief.

...happy thoughts...

[Tom]

I'm breaking Rule 1.  Although none of the work takes more than a few milliseconds in my app.  The only reason there's multiple threads is because whenever an event happens, the event handler is started on its own thread.

From what I saw of select(), if you set the timeout to 0 then when it returns it is no longer waiting for a signal from the hardware.  It might be that what happens is a new thread is spawned that sits and waits for response from the device.

The way it works on unix, is it just checks the status flags of the file descriptor (everything is a file of some sort on unix!), and then returns the fd_sets through the three arguments, one each for fds that are ready for reading, writing, and one that have an error condition (ie: it was closed on the other end). With a non zero timeout specified the process will tell the OS to wake it up when any events come in for the given fds or the timeout expires, then go to sleep. No threading involved. At least not in the way I think you're thinking.

Generally devices have hardware interrupts, when data comes in it will fire an interrupt to the operating system causing the driver to wake up and handle the data, which will get fed to the stack above, and should any processes be listening/selecting on a fd thats attached to said device, it will be notified, and woken up if need be.

[Thorin]

You're threading hardware calls? Madness!

When using the SerialPort class in .NET, you specify a method to handle the DataReceived event.  From MSDN:

The DataReceived event is raised on a secondary thread when data is received from the SerialPort object. Because this event is raised on a secondary thread, and not the main thread, attempting to modify some elements in the main thread, such as UI elements, could raise a threading exception. If it is necessary to modify elements in the main Form or Control, post change requests back using Invoke, which will do the work on the proper thread.

So I'm doing it specifically as Microsoft says it should be done - all work in the form is being marshalled back to the main UI thread.

Still had to read through that site to make sure there weren't any threading gotchas with Invoke(), though.

[Mr. Analog]

You're threading hardware calls? Madness!

When using the SerialPort class in .NET, you specify a method to handle the DataReceived event.  From MSDN:

The DataReceived event is raised on a secondary thread when data is received from the SerialPort object. Because this event is raised on a secondary thread, and not the main thread, attempting to modify some elements in the main thread, such as UI elements, could raise a threading exception. If it is necessary to modify elements in the main Form or Control, post change requests back using Invoke, which will do the work on the proper thread.

So I'm doing it specifically as Microsoft says it should be done - all work in the form is being marshalled back to the main UI thread.

Still had to read through that site to make sure there weren't any threading gotchas with Invoke(), though.

Granted my experience with hardware/GUI interaction is limited but I think there are some parallels between the way web GUI can interact with a web service call.

If you kick off a new thread every time the user hits the big red button that results in another child process that affects what the big red button does (database update, GUI update, etc) it will lead to trouble when the user starts jamming on it (or if there are communication problems).

Again, for what I do, I exclusively use/create async threaded processes on operations that are only going in one direction, as in, even if they fail we don't care (I call it "fire and forget"). A lot of that is creating requests that end up in queue where the actual processing is handled.

Err anyway, back on topic, I don't care what Microsoft says about safety there, if either one of those thread pools becomes filled or locked (for any reason) the whole transaction is borked and the main worker thread will lock, incoming requests will pile up and ka-blooey!

(err, that is to say, if no more child threads can be created, the current main worker thread will enter wait mode and pass processing to the next main worker thread, if the problem is that there is no more available resources [or whatever] the same thing will happen until you run out of main worker threads in your main thread pool, then ya can say g'night Gracie and hope there's an easy way to kill all the in proc workers with minimal cost to data integrity).

Either that or all this DayQuil has finally gotten to my brain...

[Thorin]

See, that's why I said it's hurting my brain...  Even when you think you have things figured out, there's still that lurking uncertainty.  Threading is so cool yet so hard to do 100% right.  It's almost like real security, in that sense.

Your description of using asynchronous threads makes me think of the discussion with That Guy At Questionmark about CQRS (Command Query Response Segregation) where basically there are asynchronous updates to the database that do not need to return any data.  And that hurts my brain, too.