Add an HMI to Your Real-Time Application (without adding hardware)

Whether you are designing a precision laser cutting machine or writing code for a high-speed semiconductor test system, the bottom line is that many engineering applications have similar requirements. Execution needs to happen in real-time with deterministic timing and reliable results. For years, engineers have been making use these of hard and soft real-time systems to get the job done.

The challenge is that real-time systems, by their very nature, do not typically provide the graphics capabilities needed to provide operators or engineers with information on how the system is performing. While some applications can be deployed and left running in a black-box, systems that require a human machine interface (HMI) leave the designer with little choice: add more hardware.

Traditional real-time systems often feature two computers: one for real-time processor and another to provide graphics (HMI) capabilities.

In many traditional systems, an entire second computer (whether it be single-board or standalone) must be added to a system just to provide HMI capabilities. While this method has been proven as a solution, it means added system cost,  increased footprint, and some redundant hardware. Who needs two computers that only use half of their processing power?

Virtualization technology provides a way for engineers (who are willing to slightly increase the complexity of their system) to add an HMI to their real-time applications without extra hardware. While desktop virtualization software like VMWare Workstation won’t provide the real-time performance needed for engineering applications, there exists a second type of virtualization software called bare-metal that does not comprimise determinism. That’s good news for engineers working on reducing the cost of building and maintaining their real-time systems.

Bare-metal virtualization software enables engineers to combine real-time processing and an HMI on the same computing hardware.

By installing bare-metal virtualization software, a real-time OS, and a general purpose OS with built-in graphics capabilities (such as Windows or Linux), engineers that are willing to spend some extra configuration time can build HMI-enabled real-time applications while leaving the extra hardware behind.

For more information on real-time virtualization, visit ni.com/virtualization or post your question here. How are you using virtualization in your engineering application?

 

Virtualization In Your Pocket: A Sneak Peek at Cell Phones of the Future

When most of us think about virtualization, visions of large server farms or PCs immediately pop into our heads. After all, virtualization has been used for years by IT professionals attempting to pack many different applications (often written for different operating systems) onto as few computers as possible. Applications like VMWare have also made virtualization on the PC popular, whether users are running programs written for Linux on a Windows PC, or just running multiple copies of the same OS on one computer for testing software.

The truth is that many of us may soon be using virtualization without even knowing it. As one example, several cell phones are now being developed that incorporate virtualization to run  multiple operating systems – all inside of your pocket.

At this year’s Mobile World Congress, VirtualLogix showed off their VLX hypervisor software running both Linux and a real-time operating system (RTOS) on a single phone. Specifically, an ST-Ericsson cellular system and an ARM 926 processor were paired with the VLX hypervisor software to create a very, very cool phone capable of taking advantage of both real-time OS and general purpose OS capabilities.

Why use virtualization on a cell phone? With intense competition in the cellular arena, developers can gain an edge by using high level operating system APIs to program applications and graphics quickly. Making use of so called “open operating systems” means that developers can make use of their entire ecosystem of services and existing code base without the extra work of duplicating or porting this code.

At the same time, you probably don’t want your cell phone depending on a general purpose OS to handle call processing. The timing behind sending and receiving information can be tight, and using a hard real-time operating system helps ensure that this timing is met and data  isn’t lost.

Benefits of cell phone virtualization (courtesy of virtuallogix.com)

Running virtualization software on cell phones means taking advantage of both a real-time operating system and general purpose OS on the same hardware, which can potentially reduce hardware cost and footprint. In other words, developers can have their cake and eat it too. In closing, I should  mention that VLX isn’t the only virtualizaiton software being run on cell phones. Hypervisors such as the OKL4 Microkernel have also been used to bring the advantages of multiple operating systems to a phone near you.

Answering a call while watching a video and playing a game of solitaire have never seemed so sweet.

 

Clearing the Confusion: SMP vs ASMP

As embedded systems grow more complex, there is a growing debate on how systems with more than one processing unit should be constructed. In this post, I will attempt to explain the two major approaches that have been proposed (SMP and ASMP), and clarify some common misconceptions in the process.

One hardware approach, called symmetric multiprocessing (SMP), calls for many identical processors that all execute code in parallel. In this type of a system, no individual processor has special access to memory, I/O or anything else.

Example Symmetric Multiprocessing (SMP) Architecture (http://ohlandl.ipv7.net/CPU/ASMP_SMP.html)

On the other hand, using an asynchronous multiprocessing (ASMP) approach in hardware means using different types of processors, or even different processing units such as FPGAs, DSPs, and microcontrollers all in one system. For example, National Instruments (the company I work for) offers a product called the Single Board RIO that features an FPGA and PowerPC processor that work together according to a user’s application design.

Asymmetric Multiprocessing (ASMP) Architecture (http://ohlandl.ipv7.net/CPU/ASMP_SMP.html)

Is that it? SMP means identical vs different processors? Not quite – what we have discussed thus far is only half of the puzzle. While the terms SMP and ASMP often refer to hardware (as above), the terms are also used to define software systems. As you might expect, SMP and ASMP take on different meanings in this case.

From a software perspective, SMP means programming without considering which processing hardware will be running the code that you are writing. For instance, many programmers simply split sections of code into threads and then allow an operating system to dynamically determine which threads run on various processor cores. Most x86 programmers today use this model to create parallel application software.

There are, however cases where software SMP has limitations. For example, consider a given application that only has a limited number of parallel paths (say 8 for example). In a system with 16 processor cores, many are likely to sit idle without a different approach. In addition, there are cases where designers that are making use of symettric hardware wish to customize which code is running on each processor. Perhaps one processor core is configured to gather data from I/O, while another core processes the data and a third core saves to disk.

This is where software ASMP comes in. As the number of cores on a chip rises, the need for a standard approach to programming those cores (today’s threads) becomes increasingly apparent. Therefore, symmetric hardware is likely here to stay in many commercial processors. Still, engineers that wish to create ASMP systems have several options. One solution is to configure the processor affinity of a certain thread with OS calls, or using a high level language such as LabVIEW (e.g. the LV Timed Loop). Using this method, a programmer can map different sections of code to processor cores when it makes sense to do so. An even more interesting solution is virtualization. With virtualization, an engineer can specify a group of cores to run one operating system, and another group of cores to run a second (or third, or fourth) OS. In many cases, this allows designers to make better use of symettric hardware than possible with software SMP solutions.

In short, even though the processing hardware that you are using may be symmetric, you can create powerful ASMP systems in software using techniques such as setting processor affinity and technologies like virtualization. Understanding the differences between hardware and software SMP and ASMP and knowing when to use each method (see a future post) are key to designing any parallel processing system.

 

Three Key Techniques Behind Virtualization

With hundreds of pieces of virtualization software on the market, understanding virtualization can at first seem like a daunting task. However, engineers should be relieved to know that under-the-hood most virtualization solutions really make use of the same three basic techniques. Understanding these three underlying techniques can help shine some light on how virtualization is actually accomplished in hardware or software.

Three Key Techniques  Behind Virtualization

1) Binary translation: when operating systems are running in parallel (virtual machines), they can not be allowed to alter system state in any way that would affect other virtual machines. Therefore, when a virtualized OS tries to execute a privileged instruction (such as I/O access), the virtual machine monitor can alter instructions on-the-fly so that system state isn’t affected. Basically, binary translation relies on software to catch any instructions that would cause problems in a virtualized system.

2) Hardware Assist. As you might expect, hardware assist relies on hardware rather than software to prevent virtualized operating systems from altering system state. Using hardware assist, any time a privileged instruction is executed the processor can use built-in capabilities to automatically call virtual machine monitor (VMM) software. The VMM can then execute safer instructions that have the same result (e.g. I/O) without problems. Examples of hardware assist technology include Intel-VT and AMD-V respectively.

3) Paravirtualization. The most direct approach to virtualization is directly changing both a virtualized operating system and VMM software to work together harmoniously. In paravirtualization, a virtualized OS must be modified (a designer must have access to the source code) to make calls into the VMM whenever necessary. These calls are called “hypercalls”. Paravirtualization, though it involves some work in modifying OS code, can increase performance in a virtualized system by minimizing the number of times that VMM software must be called.

Knowing about these three techniques, and which is actually used in a piece of virtualization software is key to making good decisions regarding your application. Happy virtualizing!

 

Warning: Parallel Programming is Not a Panacea

Most programmers today are well aware of the challenges that multicore processors bring. Rather than simply relying in processor clock speeds to continually increase, application designers are now required to exploit parallelism in their programs to see a speed-up on the latest hardware. Let’s consider an example; imagine an application that performs four critical tasks (e.g. interfacing with I/O, processing data, storing to disk, and updating a user interface). On a quad core processor, we can split up the tasks between processors and theoretically obtain up to a 4x performance gain (though in reality the gain will likely be significantly less).

It won’t be long before we see 8, 16, and more core processors that are commercially available. The question is – how will we keep them busy? Let’s reconsider the example above. If just 10% of our application can not be run in parallel, then the most speed-up that we can theoretically achieve is 10x. In a 16 core machine, there is a good chance that many processor cores will remain nearly idle as our example application runs.

The point is simple – parallel programming is one very necessary step towards making use of multicore processors, but it is not a panacea. Unless you are dealing with a massively parallel problem, there is only so much that a given program can be broken up in parallel.

The obvious question remains – how can we take advantage of parallelism in hardware (e.g. multicore processors) when the applications being executed have some parts that cannot be parallelized? Some solutions are running multiple applications in parallel, or running side-by-side operating systems using virtualization.

Since virtualization technology enables running different operating systems on the same computing hardware, it can be a particularly good solution for advanced applications that make use of multiple OS services. For example, engineers may take advantage of real-time data processing and a GUI on the same PC or embedded controller. Many pieces of virtualization software (called hypervisors or virtual machine managers) offer the ability to assign groups of processor cores to specific operating sytems. With this kind of setup, one physical computer is turned into an efficient, multi-talented platform.

It is clear that virtualization is a logical next-step towards solving the multicore utilization challenge. Here’s to 80 cores by 2012.

 

8 Reasons to Virtualize Your Engineering Application

With the recent buzz around virtualization technology, the obvious question is “how does this apply to my application?” Perhaps a better initial question is “can virtualization benefit me at all?” For engineers, there are many reasons that the answer to this second question may be a resounding yes.

Eight Reasons to Virtualize Your Engineering Application

8) Run multiple legacy applications on one PC. Many engineers deal with programs written for legacy operating systems that are expensive to continually upgrade. Virtualization enables users to run legacy operating systems such as Windows 95 in parallel with newer OSs, which avoids the need to maintain multiple PCs.

7) Test alpha or beta software without modifying a known-good configuration. When testing pre-release software, developers face the risk of corrupting system files, etc. For this reason, programmers may set up test machines soley for the purpose of evaluating software without altering a development machine. Virtualization enables running multiple virtual machines (VMs) on just one PC so that evaluation software can be effectively run in a “bubble” that won’t damage host OS files, etc.

6) Support multiple operating systems. As long as the underlying processor architecture is the same, virtualizing a system opens up the possibility of running two or more different OSs like Windows and Linux on the same PC or embedded controller. Why not take advantage of the strengths of different OSs where it makes sense?

5) Increase security by isolationg system functions. Hypervisors can effectively encapsulate individual virtual machines (each running an OS) and control which resources they are able to access. In addition, if an individual VM is comprimised, the overall system can continue to run without interruption.

4) Run a real-time OS and access general purpose OS services. In the engineering world, real-time operating systems are commonly used to perform deterministic test and control. It is not common, however, for real-time operating systems to provide services such as database access. Virtualization enables a general purpose OS to run side-by-side a real-time OS so that additional services can be incorporated into real-time applications.

Virtualization can enable running real-time and general purpose OSs in parallel.

3) Run a real-time OS and incorporate a GUI. Building off of #4 above, since real-time OSs place a high priority on executing tasks in a deterministic way, additional operating systems are commonly relied on to provide user interface capabilities. Using virtualization, a general purpose OS such as Linux or Windows can provide a GUI, while a parallel real-time OS is dedicated to processing critical data.

2) Make use of your multicore processor. With dual and quad core processors currently on the market, it is only a matter of time before 8+ core systems become commonplace. Though writing parallel applications will help use this processing power, if only a small percentage of a given application can’t be parallelized then its overall speedup is limited. One advantage to virtualization is the ability to partition operating systems amongst processor cores. This could be a major trend as the number of cores on a single processor chip increases over the next several years.

Processor cores can commonly be assigned to certain operating systems.

1) Save cost and footprint. The primary reason why virtualization has been so successful in the IT domain has been the dramatic cost savings (and reduction in footprint) that it can provide. Engineers can also save on cost with virtualization, even though they are less likely to be running large server farms with hundreds of machines. Instead of using separate computers or embedded controllers for each operating system that must be used (in either a development or deployment system), it may be possible to combine everything on a single multicore machine. That’s good news in our current economy.

 

The Right Approach to Parallel Programming?

With dual and quad core processors becoming the norm in commercial computers, it is apparent that multicore technology is here to stay. The remaining challenge is how to adapt software to fit this new concurrency model. While many applications run without parallel components today, adapting software to make use of multiple threads and processor cores will be key to making full use of the 8 and 16 core computers of tomorrow.

What exactly is the right approach to parallel programming? How does  a programmer implement a given application task in the most parallel way? Unfortunately there isn’t an easy answer here, only a long list of methods that are being tried and an even longer list of approaches that have been proposed.

Personally, I do not think the lack of a concensus on parallel programming should be viewed in a bad light. We are simply at a stage in time where the programming community is trying a lot of approaches to see what works. In the future, as certain methods succeed in squeezing out the maximum performance of parallel processors we are likely to see more standardization and the benefits that come along with it.

Fundamentally, the strategies for creating multithreaded applications (and therefore making use of symmetric processor cores) include:

1) Explicitly modifying code to identify sections of code that can be run in parallel (e.g. APIs like POSIX)

2) Writing code without worrying about parallelism, and relying on a compiler to identify sections that can be run in parallel

3) Using parallel libraries (e.g. matrix multiplication) that are programmed themselves to make use of multiple threads

In working for National Instruments, I am particularly intrigued by method #2. While it is certainly advantageous for programmers to optimize their applications and exploit parallelism whenever possible, it seems that the trend needs to be towards abstracting away the details of parallel programming so that developers are free to concentrate on the problem at hand and not the underlying architecture details.

Automatic Parallelization with NI LabVIEW (courtesy of ni.com)

LabVIEW is an interesting example. Since LabVIEW relies on a dataflow paradigm, it lends itself well to parallel programming (developers essentially create an application as a flow chart). Behind the scenes, the compiler automatically identifies sections of code that can be run in parallel and maps these sections to different threads. The operating system is then left to assign threads to processor cores and handle their execution.

I am anxious to see the new ways that developers make the most of parallel processors in the future, including using new APIs such as the MCAPI  spearheaded by the Multicore Association. What programming models are you using in your designs to exploit parallelism? Chime in and share your successes (and challenges).

 

Virtualization for Dummies

In my own experience, it seems that too many people shudder when they hear the word virtualization. In the absence of what the term actually means, some seem to evoke images of 3rd order differential equations straight out of a nightmarish lecture hall. I’m not going to lie – virtualization is an impressive sounding term. But the truth is, there is nothing to be afraid of.

Virtualization is just another word for abstraction. It’s that simple. The entire goal of virtualization is to remove the direct relationship between computer hardware and software (namely operating systems). When one can achieve that, a lot of fun an interesting possibilities come to light.

For example, if a single operating system (such as Windows) doesn’t “own” the hardware in a computer, one can run multiple copies of Windows all on the same machine. Or, we can take things a step further and run different operating systems combinations like Linux/Windows . With this kind of a setup, it would be possible to host a Linux-based web server while editing a PowerPoint presentation in Windows. Another useful scenareo is running a tested copy of Windows XP alongside the new Windows 7 beta to try it out.

Windows XP Virtualized on an Ubuntu Host OS (courtesy of shaymus022 on Flickr)

Think of virtualization technology as teaching your processor how to walk and chew gum at the same time. By employing virtualization, you are empowering your processing hardware to perform a wider variety of tasks than typically possible.

In reality, the concept of virtualization goes far beyond operating systems and applications on a standard PC. As an engineer, I am excited about new possibilities for running multiple operating systems on embedded devices as well. From running real-time operating systems in parallel with general purpose OSs like Windows, to running multiple independent operating systems on military machines with a high level of security, it seems like I hear a different use case for virtualization daily.

There is another reason for virtualization that has lead to it being widely accepted in the information technology (IT) world. If we can make PCs or other processing platforms multi-talented (by running multiple operating systems in parallel), we can reduce hardware requirements. Imagine that two computers are sitting in a room and each is only being used at 50% capacity. Often, the sole reason for having these two computers is to run different operating systems. Virtualization enables us to follow common-sense and combine the two computers, therefore saving cost and space.

So how does it all work under the hood? I’ll save that for another post. Please feel free to post your questions or other useful ways of thinking about virtualization below. Kudos to the most creative virtualization analogy that is brought forth.