OpenGL and Colours

Often times, in OpenGL, I tend to think about tinting colours.  Doing colour arithmetic.  Then the painfully obvious realization came to me.

I normally treat an RGBA vector as a point in 4-space.  Then why don't I normally define colour transformations through matrix operations?  Making a monochrome image becomes re-orienting (transforming) the space of colours to that of a single line running from black to white.

Even wikipedia has the matrices to go from RGBA to YUV space.  Continuing along that logic, I could build matrices that take in an image in RGBA and increase the Y component of it.

Even further along, if I use 5x5 homogeneous transformation matrices then the values can be shifted.  So blacks can become red...  Actually, this is very nicely done using SIMD operations (128-bit vectors, I think processors sporting the new 256-bit vectors are out or will soon be out.  I pray my code is compiler friendly enough to not need to explicitly worry about the issue).

Rotations do as they are expected.  For example, rotating from red to blue would continually swap the colours.  Potentially a very elegant way to manipulate colour.

Normally, interpolating of values between transformation matrices can be a bad idea.  The vectors within the matrix represent the visible space.  (imagine a 3x3 matrix.  When drawn, the resulting area corresponds to how the matrix will alter points passed to it.  Swapping the x-y coordinates is as simple as swapping the first and second rows of a 3x3 matrix when using column vectors.)  Interpolating them can cause a collapse (go to vector 0) of the space.

Obviously, there are OpenGL extensions to support this.  Some of it in software (writing the shader code or the software code to do such transforms is trivial).  I just thought of putting this out as glColor alone is very restricted compared to what can be done with matrices.  Especially since all intuitions from linear transformations in space can be applied to colours.

Insanity

Here's my take on what insanity is.  It is due to a failure somewhere in the thought process.

I used to think it was something global.  As though everything goes wrong; a mind's capabilities that end up collapsing.  Nothing legit could come out of an insane person.

But, spending time with a person that may be deemed insane at times I must revise my conclusion.  First, this insanity is of a part of the mind.  In the case I saw, it was the ability to retrieve data - the memory was dying.  Interesting may be a sick word to use in context, but I'll admit that the reasoning seemed to be working perfectly.  Operating on imperfect material just completely messed up the reasoning.  Add in very short-term memory, and it's chaos.

Another thing I noticed was slipping in and out of the dream world.  Given the person was tired, this was expected.  What wasn't expected was the content within the dreams and reality to be indiscernible.  Awaking from a dream thinking that what happened in the dream applies to reality.  I guess the opposite is true as well.

Then there was complete denial of the insanity.  All the checks and balances in the person's mind told them their logic was flawless.  Even though information was incorrect, this was not detected.  Even the fatigue did not give them a hint that they might be not thinking straight...

The fatigue also added a slowness to actions.  Seconds were needed to reason about simple things, such as how to use an appliance.

Not something to wish onto anyone.  But very annoying since the person losing their marbles can not detect themselves that they have lost it.  Hurting others in their environment without realizing it...  Animosity builds within relationships...

In the end, the madness was due to lack of sleep and lack of nutrition.  Two important things, without which we may become a burden onto the world.

Parallel Processing For the Masses?

Full well I realize that Amdahl's law specifies that the upper bound of time equals the sum of the serial portions of an application.  Another part of me wonders if it really matters.

Let me rephrase my thoughts: certain portions of applications explicitly need the extra compute power to run.  Image processing, audio processing, and physics simulations are heavy.  But the glue between these is not.

Of course, we are heading towards the land of highly parallel architectures.  There is no denying it.  Compute-intensive applications will benefit.  The OS will become more complex.  Most programmers will not see the difference.  They should not need to care about threads.  Not even about mutex.  Even less about race conditions.

Before I was advocating implicit parallelization through complex objects that would run asynchronously to the main application.  But then I started thinking about actual applications to realize that only certain frameworks would need to behave like this.  To apply any parallelization strategy globally is silly.  Consider processor affinity: a single application using a single thread on a single core will more easily benefit from the cache than a system that is distributed among cores.

If I were to write an application, my first reaction is not to parallelize it, but to see how well it behaves on a single core.  Test it.  Debug it.  Get it working.  If the performance is good enough, then I don't bother going the extra step.  The argument that processors will become more parallel at the expense of speed is valid -- my reply is that the thinking IBM's Cell was well ahead of its time.

Let us now consider a simple forms-based application.  The application may be serial, but the database engine is a highly parallel system capable of efficiently managing unthinkable amounts of data.  The form should run on a single core.  There should have no mutexes.  No race conditions.  The bottle-neck should be accessing and processing requests to the database.  To replace what is currently threads (thinking of .NET with a thread stalled with the database query while the other does the UI) the database API should be asynchronous and trigger events on the main thread of the application.   Congratulations!  Many more people are taking advantage of parallel architectures without becoming parallel programmers!

A more complex example: an image viewer.  Something that shows pictures captured from a camera.  This application will ask the OS to load an image.  Should loading the image be a sequential task?  Not for JPEG.  Should the application try to load a different image on multiple threads or should the OS work on loading images in the background and present results when needed?  The latter should be the case.  Ideally, the programmer never has to worry about a mutex.  Not even a race condition.  The word "parallel" should never come to mind.  Essentially, effort should be expended on the UI rather than the technological merits.

I've come up with a counter-example for myself.  What if an image is loaded and immediately drawn somewhere?  I'll remind myself that OpenGL is a successful asynchronous API, and that drawing only occurs when glFinish is called (maybe even glFlush, and there are other conditions, but we don't need to explore them).  So we request an image to be loaded.  That happens in the background.  Need to blit it?  Sure!  That can be added to the queue.  Display to the screen?  That can be queued as well!  At the end of the draw call tree for the window and its controls?  Good, crunch through that drawing in parallel and display it.

Let us suppose that the application also does facial recognition.  Something heavier.  Let us suppose that the OS does not support it and it must be done from scratch.  "Scratch" is never the case.  At a certain point the Eigenvalues will most likely need to be calculated for the light-corrected faces.  That sounds like something any scientific library will provide.  And these libraries will be tuned for modern parallel machines.

What I'm getting at is that parallelism is not something that everyone will have to deal with.  And no-one should want to deal with it.  Small pockets of people will deal with it.  Most coders will work in software will then become highly sequential.  A push to make parallelism exposed in all wakes of software is not the right move.  Rather, specialized libraries and software as a service should be parallel.

Software as a service?  Think an SQL server.  Indexing services specifying what files change.  Any request to the OS.  Pattern recognition services in the OS.  This opens the door to NUMA-style architectures which partition memory regions with processing units -- a much easier way to scale up the number of cores.

Pushing the idea further; parallel applications should be (in the worst case, like) servers independent of the UI.

What I'm getting at is that there are already many things that need to be thought of when writing code.  UI design and usability.  Correctness.  Resource leaks (lingering connections to a database, for example).

Performance from multiple cores, when done right, requires careful management of data.  Processor affinity must be respected.  Cache lines should not be shared amongst cores.  Cache lines should be accessed sequentially.  Race conditions.  Synchronization.

Consider that an application made to use cache correctly can run twice as fast as the same application made to use multiple threads.

Why should a controller, that ran well on a i386, be forced to worry about parallelism on multiple cores?  Are we that bad at creating application programming interfaces?

My conclusion is that parallel processing is not for the masses.  It should be available; but used with discretion.  This rush towards parallelism in all matters is part of a techno-freak's fantasized reality-distortion field.  Yes - we are going parallel.  No - we don't need to make software thousands of times more complicated to write (saving time & money) to benefit from it.

Who decides whether your average person must become a parallel programmer?  API designers.  I'm sure that if they took a step back, then they would realize that for most purposes asynchrony on a single thread is the easy the way forward for end-users of the API.

EDIT: I initially wrote this quite late.  I cleaned up the argument (a bit).

NOTE: I really should add notes about the GPU; but that will be left for another post.