This automation industry quiz question comes from the ISA Certified Control Systems Technician (CCST) program. Certified Control System Technicians calibrate, document, troubleshoot, and repair/replace instrumentation for systems that measure and control level, temperature, pressure, flow, and other process variables. Click this link for more information about the CCST program.

If a controller’s output increases when the controlled variable increases, the controller is said to be:

a) in a reset windup condition
b) direct acting
c) reverse acting
d) in a feed-forward control scheme
e) none of the above

In the ISA Mentor Program, I am providing guidance for extremely talented individuals from countries such as Argentina, Brazil, Malaysia, Mexico, Saudi Arabia, and the USA. This question comes from Caroline Cisneros.

Negative resistance also known as positive feedback can cause processes to jump, accelerate and oscillate confusing the control system and the operator. These are characterized as open loop unstable processes. Not properly addressing these situations can result in equipment damage and plant shutdowns besides the loss of process efficiency. Here we first develop a fundamental understanding of the causes and then quickly move on to the solutions to keep the process safe and productive.

Caroline Cisneros’ Question

What causes processes to be unstable when controllers are in manual?

Greg McMillan’s Answer

Fortunately, most processes are self-regulating by virtue of having negative feedback that provides a resistance to excursions (e.g., flow, liquid pressure, and continuous composition and temperature). These processes come to a steady state when the controller is in manual.  Somewhat less common are processes that have no feedback that will result in a ramp (e.g., batch composition and temperature, gas pressure and level). Fortunately, the ramp rate is quite slow except for gas pressure giving the operator time to intervene.

There are a few processes where the deviation from setpoint can accelerate when in manual due to positive feedback. These processes should never be left in manual. We can appreciate how positive feedback causes problems in sound systems (e.g., microphones too close to speakers). We can also appreciate from circuit theory how negative resistance and positive feedback would cause an acceleration of a change in current flow. We can turn this insight into an understanding of how a similar situation develops for compressor, steam-jet ejector, exothermic reactor and parallel heat exchanger control.

The compressor characteristic curves from the compressor manufacturer that are a plot of compressor pressure rise versus suction flow shows a curve of decreasing pressure rise for each speed or suction vane position whose  slope magnitude increases as the suction flow increases in the normal operating region. The pressure rise consequently decreases more as the flow increases opposing additional increases in compressor flow creating a positive resistance to flow. Not commonly seen is that compressor characteristic curve slope to the left of the surge point becomes zero as you decrease flow, which denotes a point on the surge curve, and then as the flow decreases further, the pressure rise decreases causing a further decrease in compressor flow creating a negative resistance to a decrease in flow.

When the flow becomes negative, the slope reverses sign creating a positive resistance with a shape similar to that seen in the normal operating region to the right of the surge point. The compressor flow then increases to a positive flow at which point the slope reverses sign creating negative resistance. The compressor flow jumps in about 0.03 seconds from the start of negative resistance to some point of positive resistance. The result is a jump in 0.03 seconds to negative flow across the negative resistance, a slower transition along positive resistance to zero flow, than a jump in 0.03 seconds across the negative resistance to a positive flow well to the right of the surge curve. If the surge valve is not open far enough, the operating point walks about 0.5 to 0.75 seconds along the positive resistance to the surge point. The whole cycle repeats itself with an oscillation period of 1 to 2 seconds. If this seems confusing, don’t feel alone. The PID controller is confused as well.

Once a compressor gets into surge, the very rapid jumps and oscillations are too much for a conventional PID loop. Even a very fast measurement, PID execution rate and control valve response can’t deal with it alone. Consequently, the oscillation persists until an open loop backup activates and holds open the surge valves till the operating point is sustained well to the right of the surge curve for about 10 seconds at which point there is a bumpless transfer back to PID control. The solution is a very fast valve and PID working bumplessly with an open loop backup that detects a zero slope indicating an approach to surge or a rapid dip in flow indicating an actual surge. The operating point should always be kept well to the right of the surge point.

For much more on compressor surge control see the article Compressor surge control: Deeper understanding, simulation can eliminate instabilities.

The same shape but with much less of a dip in the compressor curve, sometimes occurs just to the right of the surge point. This local dip causes a jumping back and forth called buzzing. While the oscillation is much less severe than surge, the continual buzzing is disruptive to users.

A similar sort of dip in a curve occurs in a plot of pumping rate versus absolute pressure for a steam-jet ejector. The result is a jumping across the path of negative resistance. The solution here is a different operating pressure or nozzle design, or multiple jets to reduce the operating range so that operation to one side or the other of the dip can be assured.

Positive feedback occurs in exothermic reactors when the heat of reaction exceeds the cooling rate causing an accelerating rise in temperature that further increases the heat of reaction. The solution is to always insure the cooling rate is larger than the heat of reaction. However, in polymerization reactions the rate of reaction can accelerate so fast the cooling rate cannot be increased fast enough causing a shutdown or a severe oscillation. For safety and process performance, an aggressively tuned PID is essential where the time constants and dead time associated with heat transfer in cooling surface and thermowell and loop response are much less than the positive feedback time constant.

Derivative action must be maximized and integral action must be minimized. In some cases a proportional plus derivative controller is used. The runway response of such reactors is characterized by a positive feedback time constant as shown in Figure 1 for an open loop response. The positive feedback time constant is calculated from the ordinary differential equations for the energy balance as shown in Appendix F of 101 Tips for a Successful Automation Career. The point of acceleration cannot be measured in practice because it is unsafe to have the controller in manual. A PID gain too low will allow a reactor to runaway since the PID controller is not adding enough negative feedback. There is a window of allowable PID gains that closes as the time constants from heat transfer surface and thermowell and the total loop dead time approach the positive feedback time constant.

Figure 1: 1 Positive Feedback Process Open Loop Response

Positive feedback can also occur when parallel heat exchanges have a common process fluid input each with outlet temperature controller(s) with a setpoint close to the boiling point or temperature resulting in vaporization of a component in the process fluid. Each temperature controller is manipulating a utility stream providing heat input. The control system is stable if the process flow is exactly the same to all exchangers. However, a sudden reduction in one  process flow causes overheating causing bubbles to form expanding back into the exchanger causing an increase in back pressure and hence a further decrease in process flow thru this hot exchanger.

The increasing back pressure eventually forces all of the process flow into the colder heat exchanger making it colder. The high velocity in the hot exchanger from boiling and vaporization causes vibration and possibly damage to any discontinuity in its path from slugs of water. When nearly all of the water is pushed out of the hot exchanger, its temperature drops drawing feed that was going to the cold heat exchanger that causes the hot exchanger to overheat repeating the whole cycle. The solution is separate flow controllers and pumps for all streams so that changes in the flow to one exchanger do not affect another and a lower temperature setpoint.

To summarize, to eliminate oscillations, the best solution is a process and equipment design that eliminates negative resistance and positive feedback. When this cannot provide the total solution, operating points may need to be restricted, loop dead time and thermowell time constant minimized and the controller gain increased with integral action decreased or suspended.

Q. How would you describe the book?

A. It’s an up-to-date work on control valves that presents theoretical and practical information in an easy, conversational style, making it an excellent reference for experienced instrument and process engineers as well as for students.

The book begins with a basic explanation of the function and purpose of control valves, explaining the various types of valves that are available along with their features and limitations. It also serves as a valuable best practice guide, providing: 

• Directions for selecting the best valve for a given service and flow characteristics
• Simplified equations for sizing control valves for liquids and gases under normal and special conditions, such as flashing and laminar flow
• Guidelines for minimizing environmental problems, such as noise produced by turbulent or cavitating fluids and aerodynamic noise
• Solutions to dynamic instability problems 
• Methods for improving control loop stability
• A discussion on related safety issues such as “fail-safe” action and cybersecurity

Q. What makes this book different than other books on the subject? What differentiates it?

A. As of today, there are no significant, up-to-date publications on control valves. Technology has progressed since the last publication of my books, and new knowledge has been gained in the areas of fluid mechanics and acoustics relating to control valves.

Extensive coverage of undesirable phenomena, such as liquid cavitation or excessive noise, is given together with equations on how to predict and how to avoid such occurrences. All technical information and equations are given in metric units beside the common US units, recognizing the international audience of control valve users. Examples are given in either system for ease of understanding the subject matter.

Q. What types of automation professionals would benefit most by reading the book…and why?

A. The book is primarily targeted to instrument engineers needing fundamental knowledge about control valves and their role in industrial process control systems. This book should help engineers developing new control valves avoid in their design adverse or destructive effects, such as cavitation or excessive sound levels. The book is also a valuable technical guide when used at colleges teaching automatic control theory and for refresher instruction.

Q. What makes it such an excellent reference manual? What makes the reference tables so valuable?

A. Reference tables are included to inform the person trying to select the correct size and type of control valve for a given service. As an example, the valve might need to be composed of stainless steel if the tables indicate the fluid to be controlled is corrosive.

The book also includes equations for sizing valves for all common types of liquids or gaseous fluids, including those for laminar fluid flow. The tables also provide vital information on valve materials, temperature ratings, and valve dimensions. Sizing information is especially helpful for sales representatives as it enables them to make the correct offering for a given valve specification.

This automation industry quiz question comes from the ISA Certified Automation Professional (CAP) certification program. ISA CAP certification provides a non-biased, third-party, objective assessment and confirmation of an automation professional’s skills. The CAP exam is focused on direction, definition, design, development/application, deployment, documentation, and support of systems, software, and equipment used in control systems, manufacturing information systems, systems integration, and operational consulting. Click this link for more information about the CAP program.

A set of new, 4-20mA gage pressure transmitters has been wired to an analog input card in an existing PLC system, and you are tasked with scaling the raw input values in the PLC logic. This analog card has the following characteristics:

• Unsigned, 14-bit analog data format
• Eight (8) channels, 0-20mA inputs
• Single-ended inputs
• No channel diagnostics
• Data stored internally in integer “counts,” from 0 up to the maximum 14-bit value

After reviewing these specifications, you determine the correct equation to program into the PLC to determine the current process values from the new transmitters is (where Engineering Units Value at 20 mA = EU100% and Engineering Units Value at 4 mA = 0 psig):

a) Analog Value in Eng Units=[EU100%-(Raw Counts-3277)] ÷ 4096
b) Analog Value in Eng Units=[(Raw Counts-819) × EU100%] ÷ 6048
c) Analog Value in Eng Units=[(Raw Counts-3277) × EU100%] ÷ 13106
d) Analog Value in Eng Units=[(Raw Counts+1637) ÷ EU100%] × 16384
e) none of the above

Suppose we want to do leak detection on a segment of a pipeline transporting a petroleum fluid from one place to another. Perhaps this segment directly connects the ends, maybe a pump or compressor station at the high-pressure end to a storage facility or another pipeline at the low-pressure end. We may know the length, L, the diameter of flow, D, perhaps some pipe wall properties such as the wall roughness, the elevation of each end relative to a common datum, and some things about the fluid.

To define this as a hydraulic segment we draw a dotted line, a “box,” around it, generally with consideration of where the various instruments we use to observe its operation are located. Let’s assume we have observations (measurements) of pressure and flow at each end; four observations in all. Further, we know that the pressure inside the pipe is significantly higher that the pressure of the environment around it. In normal operation we would expect that:

• P1, upstream pressure, is greater than P2, the downstream pressure
• Q1, upstream flow, is more-or-less equal to Q2, downstream flow averaged over time

If a leak occurs on the segment, we would expect the following things to happen:

• Flow upstream of the leak would go up
• Flow downstream of the leak would decrease
• Upstream pressure would decrease
• Downstream pressure would decrease

A detection algorithm could be based on the idea that the coherent occurrence of all four of those conditions is “sufficient” to detect a leak. Each of those conditions is independently “necessary,” but all four are necessary for “sufficiency.” Considering that criteria for leak detection we need observability of those four measurements in a time scheme that ensures we can establish they are all the result of the same disturbance (the leak event).

Over time, the leakage rate usually stabilizes, and the leaking pipeline migrates its operation to a new steady state in which all four conditions are evident. If the leak is stable so will be this new operating condition. Those are the observations with which we can do our analysis.

The enormous assumption thus far in this discussion is that the fluid is everywhere the same – homogeneous throughout the segment and at its ends. What if the pressure drop at the leak location is sufficient to flash some of the normally liquid flow into gas? That might limit flow through the leak path and it might change the nature of flow inside the pipe near the leak location from liquid to multiphase.

We may also discover that a leak hole of a particular size might support a much smaller mass flow rate than might be expected if conditions in the leak were the same as in the pipeline. The fastest the fluid can leak is sonic velocity and the density of the fluid at low pressure will be much smaller than at pipeline conditions. There will be a blog about such issues in the future.

Note that the term “coherence” snuck in here. It’s there to foreshadow an upcoming blog about discerning whether of not a series of observed events are, in fact, related – spawned from the same event.

At present, though, the discussion is on necessary and sufficient conditions and the significance we might attach to them.

Suppose all of the conditions are met except the upstream flow decreases instead of increasing? Depending on the upstream process equipment (e.g., a pump) it might be possible the flow would remain constant as opposed to increasing but there is usually no way a leak would cause it to decrease. A bit more thought might reveal the detected condition is, while similar to what could be expected from a leak, exactly what would occur with a shutdown or a decrease to a lower flow rate. The presence of the other three conditions are a necessary outcome of a leak, but without the appropriate behavior of upstream flow they are not sufficient.

Often, there is some seldom-used feature or unusual operating condition that can mimic most of what is necessary for a set of events to be detected as a leak. Such things can result in false alarms. Possibly the most capable engineer I’ve ever worked with was once challenged with determining why a largely above-ground pipeline in Alaska would produce false alarms with no discernable reason whatsoever.

To service one of these “probably false” alarms he was driving across an uncharacteristically bright and sunny North Slope and found himself thinking, “Well, at least these things always happen on nice and sunny days!” With that thought he changed his thinking to sun-sensitive issues and discovered a subtle problem stemming from some unproductive assumptions about a key pipeline measurement. The irritating and recurring problem was over by the end of the day.

Understanding how the process system and automation equipment work is crucial: sometimes there is a problem to fix, sometimes some process condition is screaming for attention, sometimes there is an unproductive or incorrect assumption about how things actually work. All that is fixable. Sometimes there is just one more process condition or issue that you need to know. When that is the case you need to make a way to observe it.

The art and science of project management gives us useful knowledge and powerful tools and techniques to successfully manage complex projects in various areas. Nevertheless, it is necessary for project managers to combine all these benefits with a deep understanding of the industry-specific features of projects.

Industrial automation projects are sophisticated and have special aspects that should be considered for the execution to succeed. This article analyzes and shares some of the following points:

• industrial automation projects as a combination of construction and programming
• the multidisciplinary nature of automation projects
• lack of time reserves
• commissioning as a significant and particular part of an automation project
• importance of team member qualifications
• scope definition
• peculiarities of the startup phase

These peculiarities may be obvious to some people, but for others they may not be so easy to identify. This is especially true for project managers who are not familiar with industrial automation. Moreover, this is definitely not a complete list of specific aspects of automation projects, but rather insight into some of them. Full comprehension of these processes makes a project manager’s life a little bit easier.

Figure 1. Aspects of automation projects

1. Combining construction and programming

Automation projects include various types of activities in different areas, but the interdependence of construction and software development deserves to be highlighted. Both are important, but there is usually a lack of mutual understanding between the construction specialists (supervisors and installation engineers) and the programmers. It is clearly not an easy task to be proficient in both of these areas, but definitely valuable for a project manager.

Neither construction nor programming should be neglected, or the project could be in danger. For instance, all benefits of a sophisticated and advanced proportional, integral, derivative control could be devalued by an electromagnetic noise in the measurement circuit due to the wrong cable installation. Another example is a control loop with an accurate sensor and precise valve actuator that is completely useless because a programmer reduced the accuracy by using the wrong variable type.

Therefore, it is very useful for a project manager to develop his or her own expertise in these areas or at the very least have one member on the project team who is experienced in both areas.

Figure 2. Automation project as a combination of construction and programming

2. The multidisciplinary project

Automation and control projects are almost always supposed to be done with intensive cooperation from a large number of disciplines, such as instrumentation, electrics, fire and gas, process, and information technology. An automation engineer should be proficient enough in areas like the instrument range, type of signals, accuracy of sensors, and loop calibration methods. Furthermore, almost all the equipment (e.g., valves, motors) are energized by electrical panels that often have their own logic, automatic protections, parameters, and set points. The members of an automation team have to know all the important details of the electrical part of the project to avoid future problems.

For similar reasons, the automation project team members should not only be in charge of the automation, but it is also necessary for them to scrutinize all other parts of the overall project. To be able to do that, they must be proficient in other disciplines that are closely related to the overall process of an automation project. Otherwise, a misunderstanding between various discipline specialists and inevitable discrepancies between different parts of the project (electrical, instrumentation, fire and gas, process and automation) could endanger the automation project.

Figure 3. Multidisciplinary nature of automation projects

3. Automation projects lack time reserves

The vast majority of automation projects are included into bigger projects (construction/revamping), and the most significant parts of the work are performed at the very final stage of the overall project. Consequently, time scheduling and time management for automation projects are extremely critical, since the costs of delay are very high.

The closer to the startup time of the project, the more pressure there is for team members. In normal practice, the person who initiates and funds the project is referred to as the “client” even for an in-house project. Unfortunately, due to human nature, clients tend to blame all delays on the most recently involved party, which is the automation team in this case. A project manager, as well as the team members, should be ready for these stressful conditions and follow these tips:

• define the scope of work as accurately as possible
• define all the prerequisites for other mandatory systems and disciplines (e.g., completeness of civil and piping works and the electrical power supply)
• all blocking points from third parties (e.g., construction readiness, delays in deliveries, lack of the required client personnel) should be recorded and highlighted to a client (as officially and formally as possible)
• identify the responsible people, who preferably would accept the project’s deliverables and who would agree upon all of the acceptance procedures
• think how to keep the team members at a good performance level in such a stressful environment. Define overtime conditions and benefits.

Of course, it is better to avoid delays if possible. But it is the responsibility of the project manager to be ready to protect the team from unreasonable accusations of failing to meet construction deadlines. Moreover, a project manager should be ready to show and explain to a client very clearly all of the reasons for delays that are not part of the automation team’s responsibility.

Figure 4. Automation projects are usually a final part of construction projects and lack time reserves.

4. Commissioning phase

This is the most significant and probably the most important part of the overall project, because during this stage almost all of the problems (mistakes in design, engineering, procurement, and installation) are discovered.

The automation project team members almost always become the most important part of the process at this stage. They are usually involved in troubleshooting with the other disciplines, and hence they have to share their time between their scope of work and solving some of the unexpected problems in other disciplines. It is important for a project manager to try to either avoid such distractions of the team members or at least get approval of an appropriate schedule with additional time available for the team members.

Some other features of commissioning are:

• Automation team members have to maintain a large amount of communication.
• Estimating the duration and the human resources required is not so easy, due to the high level of uncertainty, especially at the beginning of the project.
• It is very hard to reduce the duration of this phase simply by increasing the number of specialists.
• Team members who will be involved in the commissioning should be admitted and included in the project as early as possible (ideally at the design stage).

Considering these peculiarities helps to avoid typical problems like overloading team members with information and communication, lacking human resources at the final stage of the project, and underestimating time limits.

Figure 5. Communications of automation team during the commissioning stage

5. Team member qualifications

The cost of programmer mistakes is very high due to the time constraints of an automation project. There is simply not enough time during these projects to fix and repair the mistakes of underqualified engineers and programmers. The importance of the qualifications of the team should not be underestimated. It is better to spend money on preventing mistakes, which unqualified specialists could and most likely would cause in the later stages of the project, than spend it fixing them.

Besides all the required technical skills—such as knowledge of the specific control systems, expertise in related disciplines, and experience in the particular industry—a potential team member should be able to work under pressure and have basic knowledge of project management (e.g., ability to create and follow schedules, ability to communicate properly).

An automation team usually does not consist of a large number of members. Hence, a project manager should take a chance to establish a trustful and deep relationship with each member. This improves the team’s efficiency and motivation immensely.

If the project manager pays attention to the proper qualification and motivation of team members, he or she will avoid difficulties such as losing time for intensive software debugging, scope creep creating additions to the project, and lack of communication between all involved parties.

Figure 6. Optimal and real automation project execution

6. No project scope definition

Unfortunately, the scope definition for automation projects is often passed over as an outline. The technical specifications for the control systems often contain a lot of standard sentences and phrases (e.g., “in the most effective way,” “user-friendly interface,” “function should be automatic”) without any clear explanations of the meaning of these words. This creates an opportunity for various interpretations, and subsequently leads to misunderstandings, as well as troubles during the work submission and acceptance. Therefore, it could be very useful to define and to approve the technical specifications for systems, the scope of work, and especially procedures for acceptance as early and clearly as possible. Otherwise, a project team could face a situation where the client demands the work to be done regardless of the specifications, but no one knows what exactly it would mean, including the person that initiated and funded the project.

7. First the startup

Very often at the latest stages of the whole project, the only goal the client has is to start up, to push the “magic button.” Therefore other aspects of the project, such as safety, reliability, reporting, and the human-machine interface, are neglected or overlooked.

The most important drawback of this situation is that after a successful startup a client is usually not motivated to finish all of the remaining work quickly. Moreover, in this case the client has much more free time to study all the issues in detail, and as a result, there are many requests for modifications. The situation for a contractor is the polar opposite. He or she is looking forward to finishing the project as soon as possible. Therefore, if there is too much work remaining after a startup, it is most likely that the contractor will get stuck on the project. This is particularly true when the scope is not clearly defined, and a client generates more and more requests for modifications.

On the other hand, one of the possible benefits of this situation is that before a startup a client is much more flexible. The main objective at this phase is to finish the commissioning and startup, and not dive too deeply into the details of every specific issue. Therefore, it is a good opportunity to close as many issues as possible with as little effort as possible.

Generally speaking, it is better to avoid such situations, but if one has already occurred, then it is better to be ready and use its advantages.

Success journey

Automation projects have many more particularities than the ones mentioned here. Obviously, it is quite difficult for project managers to remember all of the points, but being aware of at least some of them could help them avoid disastrous pitfalls and mistakes. A wise, precise, and conscious approach to managing automation projects—with a clear understanding of specific features—makes the execution of the project smooth and predictable.

A version of this article also was published at InTech magazine.

This automation industry quiz question comes from the ISA Certified Control Systems Technician (CCST) program. Certified Control System Technicians calibrate, document, troubleshoot, and repair/replace instrumentation for systems that measure and control level, temperature, pressure, flow, and other process variables. Click this link for more information about the CCST program.

When looking at the raw input value of a single channel on an analog input card in a PLC, you see that the raw input is expressed in “counts” and is an integer value that varies between 0 and 8092. You learned from the PLC manual that 0 counts occur when the input signal is 0 mA, and 8192 counts is obtained for an input signal of 20mA. If a 4-20mA pressure transmitter is connected to this analog input channel, and if the pressure sensor has a range of 0-100psig, approximately how many counts should you see for a sensed pressure of 25psig?

a) 2048 counts
b) 4096 counts
c) 3277 counts
d) 6144 counts
e) none of the above

ISA’s mission statement is: Advance technical competence by connecting the automation community to achieve operational excellence. Are you aware of all the resources ISA has at your disposal? Are you taking advantage of them to help both you and your employer be more successful?

Below is a notable sampling of these valuable resources.

ISA standards

ISA is a standards development organization. Standards help you: take advantage of all the lessons learned by others, ease system implementation, increase your safety and security, lower your total costs, and more. ISA members can view ISA produced documents at no charge. Find out more at: https://www.isa.org/standards-and-publications/isa-standards/find-isa-standards-in-numerical-order/

ISA training

Different people learn in different ways. Some prefer to read books. Others prefer to attend courses. ISA offers a variety of courses both online and in person. Find out more at: https://www.isa.org/training-certifications/isa-training/

ISA on YouTube

Find free recorded webinars, event information and much more at: https://www.youtube.com/channel/UCwJIGlkTky1qzRnIc7Yum2A 

ISA video library

Select from more than 100 recorded webinars (free for members!) on many different automation topics at: https://www.isa.org/videos/

ISA division websites

Your Society dues include membership in two ISA technical divisions. There are 16 divisions in all. Have you selected the two that make the most sense for you and your employer? Each division has a web site and email listservs. (Note that some division web sites require you to log in to get full access to their information.) Divisions produce newsletters for their members, along with technical symposia. Find out more at: https://www.isa.org/membership/participate-in-a-technical-division/

ISA bookstore

Standards are not written to teach; books are written to teach. ISA books are written by industry experts, many of whom are members of ISA standards committees, as well as developers of ISA training courses. Find out more at: https://www.isa.org/standards-publications/isa-publications/isa-books/  

ISA leader resources

Are you an ISA volunteer? Do you know what you need to do, and how to do it, to be most effective? Section-, district-, division-, and Society-focused operating documents are available at: https://www.isa.org/members-corner/leader-resources/operating-documents/

Leadership training materials are available at: https://www.isa.org/members-corner/leader-resources/leader-training/

All of these materials exist to help you and your employer be more successful (i.e., safe, secure, efficient, profitable, etc.). Much of the material is free for members, and there are significant member discounts for items that must be purchased. What better reason to join!

This post is an excerpt from the journal ISA Transactions. All ISA Transactions articles are free to ISA members, or can be purchased from Elsevier Press.

Abstract: Timely and reliable sensing and actuation control are essential in networked control. This depends on not only the precision/quality of the sensors and actuators used but also on how well the communications links between the field instruments and the controller have been designed. Wireless networking offers simple deployment, reconfigurability, scalability, and reduced operational expenditure, and is easier to upgrade than wired solutions. However, the adoption of wireless networking has been slow in industrial process control due to the stochastic and less than 100% reliable nature of wireless communications and lack of a model to evaluate the effects of such communications imperfections on the overall control performance. In this paper, we study how control performance is affected by wireless link quality, which in turn is adversely affected by severe propagation loss in harsh industrial environments, co-channel interference, and unintended interference from other devices. We select the Tennessee Eastman Challenge Model (TE) for our study. A decentralized process control system, first proposed by N. Ricker, is adopted that employs 41 sensors and 12 actuators to manage the production process in the TE plant. We consider the scenario where wireless links are used to periodically transmit essential sensor measurement data, such as pressure, temperature and chemical composition to the controller as well as control commands to manipulate the actuators according to predetermined setpoints. We consider two models for packet loss in the wireless links, namely, an independent and identically distributed (IID) packet loss model and the two-state Gilbert-Elliot (GE) channel model. While the former is a random loss model, the latter can model bursty losses. With each channel model, the performance of the simulated decentralized controller using wireless links is compared with the one using wired links providing instant and 100% reliable communications. The sensitivity of the controller to the burstiness of packet loss is also characterized in different process stages. The performance results indicate that wireless links with redundant bandwidth reservation can meet the requirements of the TE process model under normal operational conditions. When disturbances are introduced in the TE plant model, wireless packet loss during transitions between process stages need further protection in severely impaired links. Techniques such as re-transmission scheduling, multi-path routing and enhanced physical layer design are discussed and the latest industrial wireless protocols are compared.

Free Bonus! To read the full version of this ISA Transactions article, click here.

2006-2019 Elsevier Science Ltd. All rights reserved.

With the increasing prevalence of high-profile cyberattacks and security breaches, these events may seem unavoidable. The consequences, however, come at a tremendous cost to businesses and consumers. More alarming is that the intent of cyberattacks has gone beyond stealing personal and financial data and now includes extortion, destruction of intellectual property, and damage to critical infrastructure.

Cybercriminals are becoming more aggressive and sophisticated in their attacks. As noted in one security threat report, exploit kits evolve “to stay one step ahead of security systems, with greater speed, heightened stealth, and novel, shape-shifting abilities.” 

Protecting critical infrastructure

In the U.S., the potential for a cyberattack on critical infrastructure is a growing concern. In the U.S., the Obama administration issued Presidential Policy Directive (PPD)-21 – Critical Infrastructure Security and Resilience. The directive states, “The nation’s critical infrastructure provides the essential services that underpin American society. Critical infrastructure must be secure and able to withstand and rapidly recover from all hazards.” Those hazards include cyberthreats. Presidential Executive Order 13636 – Improving Critical Infrastructure Cybersecurity was released in conjunction with PPD-21 to specifically deal with the cyberthreat to critical infrastructure. Per the executive order, “The cyberthreat to critical infrastructure continues to grow and represents one of the most serious national security challenges we must confront. The national and economic security of the United States depends on the reliable functioning of the Nation’s critical infrastructure in the face of such threats.”

OT security

A key component in protecting critical infrastructure from cyberattack is protecting the automated systems used to monitor and control critical processes. Systems that control water and wastewater processes are known by many names. Industrial control systems, supervisory control and data acquisition, distributed control systems, and industrial automation and control systems (IACSs) are just a few of the terms that fall under the general category of operational technology (OT). 

Attacks on OT systems and networks are becoming more common. Although some high-profile attacks on critical infrastructure have been reported, we do not know the full extent, because cyberattacks do not have to be reported unless there is a breach of personal information or financial data.

Because we do not hear of many attacks on critical infrastructure, some people assume the cyberthreat to OT is not a great concern. We might not know the actual number of attacks, but we do know that malware has been developed specifically to attack critical infrastructure (e.g., Stuxnet and Black Energy). In December 2015, for example, an attack on the Ukrainian power grid left hundreds of thousands without power. These attacks are a powerful reminder that the threat to critical infrastructure cannot be ignored.

NIST cybersecurity framework

Executive order 13636 directed the National Institute of Standards and Technology (NIST) to develop a cybersecurity framework to reduce risk to critical infrastructure. The intent of the framework was to provide critical infrastructure owners and operators a flexible and repeatable approach to meeting baseline cybersecurity measures and controls. In February 2014, NIST released its Framework for Improving Critical Infrastructure Cybersecurity Version 1.0. The framework is available at www.nist.gov/cyberframework.

The cybersecurity framework (CSF) is a voluntary, risk-based approach for managing cybersecurity risks for critical infrastructure. It references industry standards, guidelines, and best practices known as informative references to help organizations manage cybersecurity risks.

The water sector does not currently have specific directives for securing OT, so the CSF is a useful resource for identifying relevant resources. The CSF is not meant to replace an existing program, but can be used as the foundation for a new cybersecurity program or to improve an existing program. The framework consists of three parts: the implementation tiers, the framework profile, and the framework core (figure 1). 

Figure 1. NIST Cybersecurity Framework

Framework implementation tiers define the organization’s risk management practices by one of four tiers. Tier 1 represents the least amount of risk management, and tier 4 the most. Each organization must determine which tier is appropriate for it, given the organization’s unique goals, feasibility of implementation, and acceptable level of cybersecurity risk.

The framework profile helps an organization define a road map for moving from a “current” profile that defines current risk management practices, to a “desired” profile that defines the outcomes needed to achieve the desired cybersecurity risk management goals. Comparing the current profile to the desired profile produces a gap analysis that can be used to establish a plan defining actions required to meet organizational goals, and to prioritize activities for cost-effective allocation of resources. 

The framework core is a set of cybersecurity activities, desired outcomes, and applicable references common across all critical infrastructure sectors. They are segmented into five functions (figure 2). These functions organize basic cybersecurity activities at their highest level. The five functions are identify, protect, detect, respond, and recover.

Figure 2. The framework core

Figure 3 shows how a function (identify) is broken down into various categories (asset management for this example). Categories are broken down into subcategories (physical devices and systems inventoried), leading to specific informative references, such as the ISA-62443 standard. Additionally, the specific section of the informative reference associated with the subcategory is provided to clearly identify the content most relevant to that subcategory. 

The informative references listed by the CSF are not mutually exclusive, but complement one another. One resource is likely to provide more detailed guidance than another on a particular aspect of cybersecurity. As a result, all relevant resources should be considered when developing or updating a security plan.

ISA-62443 is one of the key standards referenced in the CSF. ISA developed this multipart standard for OT security. The standard provides a flexible framework for developing a comprehensive security plan for critical infrastructure entities such as water and wastewater utilities. 

Figure 3. Linking cybersecurity function to informative references

One particularly important section is ANSI/ISA-62443-2-1, Security for Industrial Automation and Control Systems Part 2-1: Establishing an Industrial Automation and Control Systems Security Program, which is aimed at asset owners and operators responsible for establishing and managing a utility’s cybersecurity program. Unlike other security standards that cover only technical considerations for cybersecurity, ISA-62443-2-1 focuses on the critical elements of a security plan relating to policies, procedures, practices, and personnel. It is a valuable resource to management for establishing, implementing, and maintaining a utility-wide security plan.

The first step in developing an OT security program as defined by ISA-62443-2-1 is risk analysis, starting with the business rationale for cybersecurity. As noted in the standard, “Establishing a business rationale is essential for an organization to maintain management buy-in to an appropriate level of investment for the IACS cybersecurity program.”

Why a business case?

A well-defined business case for automation cybersecurity is essential for management buy-in to ensure the long-term allocation of resources needed to develop, implement, and maintain a utility-wide cybersecurity program for the OT controlling critical infrastructure. Without a strong commitment by senior management, utility personnel will find it difficult to prioritize the allocation of resources—especially when faced with resource-intensive challenges such as aging infrastructure.

The business rationale for cybersecurity is based on the potential impact that a cybersecurity event can have on public health and safety, the environment, business continuity, emergency preparedness, regulatory compliance, and the public’s confidence in the utility. Developing a business rationale for cybersecurity identifies the business reasons for investing in cybersecurity to lower risk and protect the utility’s ability to perform its mission.

Cybersecurity is not an absolute, but a matter of degree. Because most water/wastewater systems have limited funding and personnel, mitigating all threats is not feasible or practical. By defining a business rationale for OT cybersecurity, executive management can define acceptable levels of risk for the utility, so that utility personnel can better understand the priorities to address in the security plan. By determining the cost-benefit aspects of security measures, the utility will get the maximum results from the money spent. Not having a well-defined security plan results in inefficient use of limited resources and can create a false sense of security.

When analyzing the business rationale, executives may find economic benefits similar to those of worker safety and health programs. Each year, workplace deaths and injuries cost U.S. businesses tens of billions of dollars. The Occupational Safety and Health Administration reports that employers save $4 to $6 for every $1 invested in an effective safety and health program.

Similarly, each year security breaches cost businesses billions of dollars in fines, litigation, and lost customers. According to the Ponemon Institute’s 2016 Cost of Data Breach Study, the average cost of a data breach is $4 million, a 29 percent increase since 2013. However, an attack on a critical water or wastewater system could have significant consequences that far exceed the monetary costs. A cybersecurity event that negatively impacts operations could expose a utility to litigation affecting business continuity and its ability to carry out its mission.

OT security is also fundamental to the creation of a culture of security within the utility, as noted in the American Water Works Association (AWWA) standard ANSI/AWWA G430-14 – Security Practices for Operation and Management. A key directive of the standard is an “explicit and visible commitment of senior leadership to security.” The AWWA G430-14 standard addresses the broad issues of security, and protecting operational technology is a key facet of security. An established business rationale for OT cybersecurity shows that management takes its commitment to security seriously. Cybersecurity must become a fundamental component of the utility’s culture, just like safety.

Should I worry about a cyberattack?

The probability of a state-sponsored cyberattack on a utility is most likely extremely low. However, water and wastewater utilities might be viewed as easy targets by radicalized, lone-wolf threat actors. “Security by obscurity” is no longer an option for small and medium utilities that have not considered external threats a concern in the past.

However, attacks make up only a small part of cyberthreats, as most originate internally. Whether malicious or accidental, the utility’s goal should be to prevent or minimize any type of cybersecurity event that will affect the availability and reliability of a critical system. A comprehensive security plan recognizes and prepares for both intended and unintended cybersecurity events. This will ultimately enhance the utility’s overall security and minimize any negative consequences on business continuity.

IT’s job?

Many in the OT world assume that their information technology (IT) department is handling the cybersecurity plan. IT professionals, who are responsible for ensuring the availability, integrity, and confidentiality of business and enterprise networks, are important members of a cross-functional team that develops and implements a utility-wide cybersecurity plan. However, the responsibility for protecting OT systems and networks—and the critical infrastructure they control—from a cybersecurity event lies with those who operate and maintain those networks.

An established cybersecurity business case will clearly define security roles and responsibilities for all utility personnel, including those involved with emergency preparedness and business continuity.

Figure 4. High-profile cyberattacks and security breaches are reminders that a comprehensive security plan is needed to protect industrial control systems and the critical infrastructure they control. A well-defined business case for automation cybersecurity will ensure management buy-in and long-term allocation of resources.

Guidance for developing a business case

A useful feature of ISA-62443-2-1 is annex A, which provides guidance on developing all elements of the cybersecurity management system as defined in the standard. Annex A includes helpful information for applying the standard and tailoring it to the organization’s specific needs. Although it is not a step-by-step process, it does offer useful guidance in developing each element, including the requirement for a business rationale.

Per annex A, there are four key components of a business rationale: prioritized business consequences, prioritized threats, estimated annual business impact, and cost of countermeasures.

Prioritized business consequences: For a water or wastewater utility, there are numerous areas where a cybersecurity event could cause significant negative consequences for operations. Those areas include public health and safety, the environment, business continuity, emergency preparedness, regulatory compliance, and public confidence in the utility to fulfill its mission. It is likely there will be compelling business reasons for ensuring that the consequences of a cybersecurity event—intended or otherwise—are not realized.

Prioritized threats: As stated previously, it is neither practical nor feasible to fully mitigate all risks. Limited resources demand that the most credible threats be given priority for developing mitigation strategies and allocating resources effectively. Unfortunately, there are many events that can negatively affect operations, from disgruntled employees to radicalized, lone-wolf threat actors, to common technical defects. By prioritizing threats in the business rationale, it will be clear which ones are considered most credible by management and which have the greatest potential impact on business.

Estimated annual business impact: The list of prioritized business consequences should be evaluated to determine an estimate of the annual business impact, ideally in financial terms. There are costs associated with implementing countermeasures to prevent or minimize a cybersecurity event. Unless there is a much larger cost to the business than the cost of countermeasures, it will be difficult to justify the cost of the countermeasures.

Cost: The purpose of the business rationale is to justify the anticipated cost of the human effort and technical countermeasures required to manage cyberrisks. The larger the difference between that cost and the estimated annual business impact, the easier it will be to justify allocation of resources. Estimating the cost of technical countermeasures should be straightforward. Estimating the cost of the human effort will be more challenging. Fortunately, ISA-62443-2-1 addresses all aspects of managing cyberrisks, including the most important one: people. The standard defines how to organize for security and provides guidance to help estimate the human effort required to manage cyberrisks.

First step

The cyberthreat scenario for critical infrastructure, including water and wastewater systems, is increasing and will become only more challenging in the future. The first step to creating a security plan is to define a business case for OT cybersecurity. By justifying the business rationale, a utility can reduce its cyberrisks, increase its resiliency, and ensure the availability and reliability of water and wastewater systems. ISA-62443-2-1 is a valuable resource for developing a cybersecurity management system that is essential for protecting critical infrastructure.

A version of this article also was published at InTech magazine.