The Truth about Cellular IP

I have been very busy analyzing real-world telemetry traffic over cellular IP, and unfortunately I am now 100% convinced that you cannot effectively use most (any?) off-the-shelf "Ethernet" software tool to talk to remote Ethernet devices over cellular IP. Bottom-line is that - unless your host app is custom written to be data cost and time delay sensitive - your data costs will be bloated due to the nature of the tool. Even something as "obvious" as adding compression doesn't solve the problem because telemetry packets tend to be too small for effective compression. For example: an 8-byte Modbus/RTU request becomes 12-bytes after ZIP-style compression. Plus this doesn't reduce the 104-bytes of TCP overhead nor 28-bytes of UDP overhead. None of the cellular providers allow use of RFC-class TCP header compression, since it requires all of the infra-structure to maintain
copies of headers etc.

So I have been working on "reduction" solutions - how to obtain the effect of moving "X" IP packets but only moving "X-minus-a-bunch" of actual IP packets.

Tunneling TCP thru UDP
The most promising and generic form of reduction is to tunnel TCP/IP via UDP/IP over cellular. So the host application talks TCP/IP to a local proxy, which acts as the TCP end-point. All of the TCP SYN, ACK and Keepalive traffic is limited to the local Ethernet. The local proxy then initiates a UDP "session" with a remote proxy over cellular & we instantly see a 60-90% reduction in data costs. The remote proxy initiates a TCP/IP connection to the remote Ethernet device, which again isolated the extra TCP overhead to the remote Ethernet.

The reaction of non-IA network engineers to this idea is predictable and a bit humorous after a while. They immediately say "You cannot do that!!! UDP/IP is unreliable!!! You'll break something!!! You are committing a mortal Sin!!!" But in reality none of the IA protocols leverage the reliability of TCP anyway. For example, Rockwell RSLogix doesn't send a program block to a ControlLogix and blindly assume it was successful after the TCP Acknowledge from the peer is processed. Instead, RSLogix sits (blocks literally) and waits for a successful CIP response on a single CIP Connection. So if the local proxy returns a TCP-ACK to the RSLogix host and the CIP request is lost within the UDP/IP tunnel ... eventually RSLogix times out the CIP connection and the application (and/or user) will restart.

Fortunately, cellular is very reliable - all of my tests sending 10,000 UDP packets rarely even lost 1 packet and I'm not sure if such a rare loss is due to cellular or just my test script hiccupping & dropping a packet. Plus cellular tends to have only very bursty error problems. In other words, you won't lose 1 packet per 10,000; instead you'll lose all packets for 5 minutes or just 5 random packets out of a group of 10 sent. This shotgun-damage tends to confuse TCP/IP state machines to the point that they abort the connection anyway. In truth, in all of my Wireshark/Ethereal trace reviewing I have never seen a single situation where a TCP retry did anything but add data cost; every TCP retransmission just results in a "Duplicate ACK" showing up a few packets below in the trace & a doubling of the cost of that block of data.

So overall, anyone planning to use cellular should first investigate if they can use UDP/IP instead of TCP/IP.

TCP Problem #1 - added cost for pointless ACK
As mentioned above, real-world analysis of telemetry use of TCP shows the TCP ACK isn't useful; but worse, Embedded TCP devices tend to sub-optimize the ACK timing to "speed up" data transmission and recovery. Almost universally moving an IA protocol via TCP/IP results in 4 TCP packets instead of the idealized 3.

• Your app sends a TCP request (request data size + 40-52 bytes of overhead)
• 800-1100 msec later your app receives a TCP ACK without data (another 40-52 bytes of overhead)
• 10-40 msec later your app receives the protocol response (response data size + 40-52 bytes of overhead)
• Within a few msec, your app sends the TCP ACK without data.

So what would have been a 2-packet transaction with only 56 bytes of overhead under UDP/IP, or what should have been a 3-packet transaction with 120-156 bytes of overhead under ideal TCP/IP usually becomes a 4-packet transaction with 160-208 bytes of overhead. Yes, there exists a TCP socket option and the concept of "Delayed ACK Timer" to prevent the first empty TCP ack from being returned over cellular, but few embedded products use this since it adds code complexity, and it slows down overall data communications. At least in the IA world it seems everyone wants their Ethernet Product costing 2-4 times more than their serial product to appear lightning fast. So they ignore the TCP community's decades of hard-earned experience and "hack" their TCP stack to sub-optimize fast local Ethernet performance.

So this is where the instant 60-90% data cost savings of using UDP over TCP comes from. UDP has smaller headers and results in fewer packets being sent. Since the cellular IP system is "encapsulating" your TCP/IP packets in a manner similar to PPP, the entire IP header, TCP or UDP header, and your data is all considered billable payload.

There is also a myth propagated to this day that the TCP ack causes retry to occur more rapidly out in the wide-area-network infrastructure. The rhetoric goes, "If the 3rd and 4th router link is congested and the TCP data packet is lost, then the 3rd router will retransmit ... which is faster ..." Perhaps this was true back in the 1980's, but today the 3rd and 4th router (and all of the other 20 to 30 routers in a cellular end-to-end path) are just tossing IP packets upstream with no awareness of the packet functions. In reality, it is only the TCP state machines within your host machine and within your remote device that have any ability to retransmit anything.

TCP Problem #2 - added cost for premature retry
The TCP RFC includes many dynamic timers that automatically adjust themselves based on real-world performance. This is actually pretty neat. It means if the TCP ACK and response times tend to be longer than normal, then the TCP state-machine slowly increases the delay before retransmission. But I've seen 3 problems with this.

1. The most effective way to leverage auto-adjust is to include the 12-byte TCP header options that time-stamps all packets. Linux system add this by default and installing one of many PLC engineering tools on your Windows computers causes Windows to also start always using this. The setting generally is global - you either have 40-byte TCP headers or 52-byte TCP headers forever. So for small telemetry packets, this adds a disproportionately large increase in data costs.
2. Many embedded devices (PLC, RTU and I/O devices) have "hacked" the TCP ACK sub-system to force connection failure to be faster than the standard 3-4 minutes. For example, I worked with one large PLC company which expected TCP sockets failure in less than 1 second, so they forced TCP retransmission in hundreds of msec and without any normal exponential backoff between retries. This is totally unusable over cellular; you will end up with 30% to 90% of your data traffic being premature retries and responses to premature retries. I have literally seen Wireshark/Ethereal traces which are mainly black lines with red text - which is the default color used to show TCP "problems" such as lost-frag, retrans, dup-ack, etc.
3. The latency in cellular is abnormal by an order of magnitude. Even browsing the internet or doing a telemetry polling test over DSL/cable broadband averages latencies in the 100-150 msec range. This is what a Windows or Linux defines as "slow/bad" - not the 800 to 3500msec of cellular. So even watching a Windows or Linux TCP state machine auto-adjust the retransmission delay over time, you will not see it achieve a 100% effective setting which eliminates wasted TCP retransmissions. The delay seems to top out
   at about 1.5 to 1.8 seconds, which is just too close to the actual "normal" latency range. So again, use of UDP/IP frees the use user from data costs associated with TCP legacy assumptions - both the main-stream MIS/IT market variety of assumptions and the misapplied IA vendors "speed-ups".

TCP Problem #3 - uncontrollable SYN/Socket Opens

Given the way all cellular systems "park" inactive cellular data devices, it is exceedingly rare to ever see a host app open a new TCP socket without prematurely retrying/retransmitting the SYN packet. This is because one is virtually guaranteed that it will take about 2.5 seconds for the data device to be given active airwave resources and return the SYN+ACK response. This has NOTHING to do with the "always connected" feature Digi and others claim. The data device (even when parked) is fully connected by IP and fully authenticated by the system - it is "always connected". However, the local cell tower only has finite airwave resources, so any device (cell phone or data device) which is idle from 3 to 45 seconds is "parked" without having any preallocated airwave resources. Literally when the TCP "SYN" shows up, the cell tower has to use the control channel to inform the data device to request airwave resources, and after these are requested and allocated the data device can receive and response to the TCP socket open request.

But that's not the real problem related to TCP Socket Opens ... the real problem is yet another case of IA vendors sub-optimizing TCP behavior for fast local Ethernet performance. For example, I once had a customer who normally paid about $40 per month receive a $2000 bill one month. It turns out they had powered down the remote site for 3+ days and the off-the-shelf 3rd-party host application they used would try to reopen the TCP socket every 5 seconds!!! So Windows would send the initial TCP SYN to start the open, since the remote was off-line Windows would retransmit this TCP SYN a few seconds later. After a total of 5 seconds, the application would ABORT this TCP socket attempt and start a new one. So this host app was pushing 24 billable TCP packets per minute out to a remote site that was powered down. This was nothing the host app vendor documented, nor was it anything a user could configure or over-ride. The user could configure the host app to ONLY poll once per 5 minutes; but the user had no control over this run-away TCP SYN/Open behavior.

Tunneling TCP through UDP effectively decouples the TCP SYN/Open from cellular data charges. The first TCP Syn/Open request to the local proxy would succeed even if the remote IP site is offline. No retries would be required. Even if the host app attempts to retry the data poll every 5 seconds, this is something the UDP proxy can be configured to "resist". If the user truly wants data packets to only move every few minutes, that is something the UDP proxy can easily enforce.

TCP Problem #4 - sub-optimized TCP keepalive

The final problem I'll discuss (but not by any means the "last" problem with TCP) is that many embedded IA devices have relatively fast TCP Keepalives hard-coded to speed up lost-socket detection. While this is an admirable goal, a Rockwell PLC sending out a TCP Keepalive at a fixed 45 second interval can create up to 6MB of monthly traffic by doing this. Siemens S7 PLC seem to issue TCP keepalive every 60 seconds - a bit better, but not by much. Maybe such a heart-beat is useful to know the remote is accessible, but given the reliability of cell phones (when the last time you had a dropped call or no signal ...) you'll obtain a lot of false-alarms if you treat every missed packets as something requiring maintenance's attention.

Again, tunneling TCP through UDP effectively eliminates the automatic, possibly uncontrollable use of TCP Keepalive. If your process can handle you talking to it once an hour, then the cost of TCP socket open and close, as well as any TCP Keepalive is all wasted investment.

Not only this, but the cellular providers do NOT want users who send a simple, rather empty packet every 30 to 60 seconds - this is literally the worst kind of customer, as this forces the cell tower to "waste" one of its very limited airwave resources with almost no income returned to the carrier. From what I hear, carriers either want customers who talk constantly and pay huge monthly fees (say $90 to $350/month); or they want customers who rare talk and pay a small fee (even just $5/mo) but cost the carrier virtually no direct expenses.

Putting this is "restaurant terms":

• A cellular data device that talks constantly but pays for a large plan is like a restaurant patron who sits at a table, constantly ordering more food and paying a larger bill.
• A cellular data device that rarely talks is like the restaurant patron who comes in once a month, sits at a table, orders a meal, pays and then vacates the table.
• A cellular data device that keeps an idle channel open full time but rarely talks is like the restuarant patron who sits at a table in the resturant, reading the paper but rarely ordering food or paying a bill.

In fact, in private chats with carrier account people, I have heard several times that they have been directly to prefer either customers who talk constantly on large plans or those who talk at most once an hour (better once a day) on small plans. Customers planning to talk every few minutes have been defined as bad investments. It may be fair to say that after years of building up the data-plan customer base, the cellular carriers have come to understand that the REAL cost of data plans is not the bulk data bytes moved; it is instead the percentage of time the device consumes (or squats on) 1-of-N scare airwave resources in proportion to the monthly fee they pay.

ODVA, Rockwell, and Modbus Get Smart

Summary: next week an amazing joining of technologies will take place at ODVA in Ann Arbor MI as ODVA, Rockwell, and Schneider-Electric discuss how to integrate Modbus servers (slaves) into CIP Producer/Consumer systems

One of the fun things about being involved in "multi-vendor" solutions is when you recognize moments of amazing sanity as they occur. One such moment of amazing sanity is occurring next week when ODVA (aka Rockwell / Allen-Bradley) and Modbus supporters (aka Schneider-Electric / Modicon / SquareD / Telemecanique) sit down to discuss how to integrate Modbus devices into the ODVA Ethernet/IP and CIP network systems. Of course there must be some interesting hidden politics behind this move - and I somewhat light-heartedly believe that perhaps French Schneider-Electric sees joining with the Americans (Rockwell/ODVA) as the lesser of two evils when compared to joining with the Germans (Siemens/PNO).

Check out: ODVA Call For Members: Modbus Integration JSIG The kick-off meeting for the Modbus JSIG runs from Thursday, May 10, 11:00 AM to Friday, May 11, 04:00 PM

Side-stepping the marketing fluff and platitudes of a brighter future such meetings evoke, small third-party suppliers and the folks on the plant floor can expect the following benefits. Regardless of the directly stated goals of ODVA, Rockwell, or Schneider-Electric, small vendors will implement solutions that include these abilities:

• Vendors making Ethernet Modbus/TCP products will have a simpler "first step" to adding full ODVA/CIP support without the somewhat overwhelming task of 100% conversion of a word-array device model into hundreds (or thousands) of CIP objects.
• ControlLogix PLC will be able to connect through Ethernet-to-Serial devices to multi-drops of Modbus/RTU slaves. For example a user with a dozen small Modbus/RTU PID loop controllers will be able to add an Ethernet-to-Serial device to read via Modbus and cyclically produce a small block of word data from each loop controller over Ethernet.
• HART, Bluetooth, ZigBee and other new technologies which offer Modbus interfaces will find a instant place as sensors and I/O within CIP and Rockwell systems.
• Since Siemens, GE-Fanuc, Omron, Mitsubishi and most major PLC brands offer some method to act as Modbus slaves, users with any of these PLC will be able to integrate them within the CIP Producer/Consumer system.
  

Looking forward a year or two, I also can foresee some other secondary benefits evolving from this JSIG's work:

• I started working with ODVA Ethernet/IP almost 8 years ago and still as-of today the legacy PLC5E, SLC5, and serial MicroLogix (the old PCCC-based PLC) don't have effective inclusion within CIP Producer/Consumer systems. Since the device model of PCCC PLC shares much in common with Modbus PLC, it is a very small enhancement to add a similar support for AB PLC - perhaps AB will actually extend this to future firmware updates to Ethernet-based PLC. In fact, since my Digi One IAP code already allows Modbus masters to query DF1 and CSPv4 slaves as-if Modbus slaves, as soon as Digi adds Ethernet-to-Modbus support per this JSIG's output users of older AB PLC will gain access to CIP Producer/Consumer systems indirectly as honorary Modbus slaves.
• Today legacy Modbus and Modbus/TCP systems lack any simple form of multicast producer/consumer exchange. While the IDA protocol offers this, IDA is so many orders of magnitude more complex (and resource hungry) than simple Modbus as to become really an "unrelated" protocol. Any specification that defines a "server interface" naturally implies a corresponding "client interface". So although this ODVA JSIG is not planning to define how Modbus "peers" could use multicast to exchange cyclic data, the end result will be a fairly natural and multi-vendor method to do this. So while I doubt many pure Modbus/TCP products would implement ODVA protocols just to gain this multicast exchange, any products which add the CIP support anyway will naturally add the last few bits of code required to enable pure Modbus-to-Modbus multicast via the ODVA mechanism.
• Taking the above point to its natural conclusion means Modbus/TCP masters which implement the Modbus JSIG's "server" function will also gain a mechanism to access CIP Producer/Consumer systems. Even if the ODVA JSIG doesn't cover how to do this, natural methods will be inferred, produced, and copied by vendors to make this a fairly common new product feature.

And lastly, it will interesting to see what response PNO considers to likewise include Modbus slaves more formally into PROFINET. Personally, I have always felt that the simpler PROFINET IO protocol would fit very naturally with multi-drops of Modbus slaves acting like plug in I/O modules ... each slave would be a module within the IO device.

Do Users Really Want Industrial Ethernet?

(For those impatent to read this to the end - I'm not saying don't use Ethernet ... I am just saying be careful you understand what your customers expect and what functionality they will assume you include *for free* when you add Ethernet)

My last post created some interesting feedback. But I want to emphasize a topic from that post more fully. For the last 15 years I've been involved in the "multi-vendor interface" business - linking multiple vendors' equipment by data comms. First I worked in RS-232 and 485, then fiber optics, then Ethernet, and now by virtually every technology that moves TCP/IP.

From time to time I am contacted by some pretty desperate customers - for example I had one customer who had piloted some Ethernet-based temperature sensors. Things worked fine in the lab with their lab computer, so they bought 50 ... only to find out they couldn't use them. It seems these sensors really were "just Ethernet" - they talked by Ethernet broadcast and direct MAC-layer packets. They didn't support TCP/IP and therefore could NOT be routed by any standard network infrastructure. The user could not talk to any of the sensors they had intended to install in panels around the plant because the "Computer Room" wasn't on the same physical Ethernet segment as the "floor". There was no way to broadcast or unicast MAC-level between the systems. This customer hoped I knew of some magic box to act as gateway between TCP/IP nodes and pure Ethernet nodes; I didn't.

So this brings me back to the concept of the true cost to implement "Ethernet". Customers who ask for Ethernet are not really asking for Ethernet hardware or an Ethernet media bus. They have the expectation that they can interface your "Ethernet Devices" with the wide variety of other equipment they have - including WiFi, routed Ethernet, fiber optics, wide-area networks, and so on. They also expect (at least in a future firmware rev) web pages for configuration, SNMP for remote management, strong encryption, and so on.

So the term "Ethernet" has taken on a life of its own - remember when 802.11 was called "Wireless Ethernet". Well, there is absolutely NOTHING Ethernet about 802.11, yet it was a useful PR move to link the two. No doubt it helped spread the acceptance of WiFi as we now call it. Interestingly enough, the current PCI verse PCI-Express adapters you buy for a PC are using the same PR trick - linking a new, unknown technology to an old established technology that merely accomplish the same function by very different means. Maybe Sony should have called Beta-Max VHS-Max instead ... but then I'm showing my age by even knowing that a consumer-oriented video standard other than VHS even existed.

But back to Ethernet. If you are a small device maker and have yet to start making Ethernet-based products, just be aware that customers who ask for "Ethernet products" aren't really asking for ... err, products with Ethernet. They are asking for products which integrate into (at a minimum) the wide family of TCP/IP based technologies out there. I am not even talking about should you support Modbus/TCP or ODVA Ethernet/IP or ProfiNet yet. I am just saying customers will expect your "Ethernet products" to be able to hold a raw TCP/IP or UDP/IP conversation with all of the other equipment they are investing in daily.

So the cost to add an Ethernet port is just a small part of your cost to "add Ethernet". That is why companies like Digi can sell Ethernet-to-Serial converters or sell "async Ethernet driver chips" like the Digi Connect ME which links to your CPU's serial UART. These devices of course cost more than $9.95 or even the cost of a few new hardware chips, but that higher cost is paying for TCP/IP, web servers, SNMP servers, strong AES encryption and all of the other things your customers expect when the buy "Ethernet products".

So to digress a bit, I suffer this "Oh, don't worry ... it's Ethernet" on a daily basis. So far I have to say at least 95% of the off-the-shelf software applications I test supporting TCP/IP don't work well with technologies other than direct Ethernet. This includes problems not only when extremely different media like satellite or cellular, but even when WiFi is used. So that is part of my mission in this blog - what you want is NOT to Ethernet-enable your products. Instead you need to "IP-enable" your products by way of an Ethernet interface.

Do you need a special Industrial Ethernet hardware?

This is a bit off topic, but someone asked me about “Industrial-Grade” Ethernet designs. Well, I’m not a hardware expert but have better than average exposure to issues of grounding and surge management. Plus I’ve been involved in many detailed discussions about how things may break Ethernet devices with engineers from Rockwell and other industrial suppliers. Bottomline is standard Ethernet designs are quite robust - the main thing I see hurting careless designers is PCB via or tracks violating the 1500v isolation of the Ethernet magnetics. This means a customer actually testing high-voltage noise or surges could see arcing across the isolation barrier which causes product reboots or even failure.

The $9.95 Mindset:
Any detailed discussion about “special Ethernet for Industry” first starts with the fact that customers can buy 10/100 computer Ethernet adapters from any big-box store for $9.95. So users have this perception that the increment for Ethernet is small & cheap. While they may not expect you to sell Ethernet products for $10 more than serial products, they won’t be happy to hear that your Ethernet product is $300 more. I will tie this together below, but the bottom line is the closer you can match your Ethernet hardware to the market norm, the lower your over all costs will be.

Who Pays for Extra Software Work?
Of course, that $9.95 computer Ethernet card doesn’t include:

1. Microsoft’s ROI on TCP/IP and network stack work
2. the OPC server cost to add Ethernet drivers in place of serial drivers
3. tool vendors need (ie: your need) to rewrite serial-based tools to become network-based tools.

Plus don’t forget that the customers are NOT really just asking for “Ethernet” – Customer X will want web pages for configuration, Customer Y will want SNMP for remote management, and Customer Z will want strong 128-bit AES encryption suitable for general US-Government usage. So unlike the $9.95 PC card, you must hope your customers help pay for a whole lot of added, ongoing engineer work.

What is the Market Supply Sweet-Spot?
Go online and look at the cost of hard-drives – a 300GB (300,000MB) drive is in the $75 range, while an old 20MB drive (Meg, not Gig) costs about $140. We all understand this oddity – there is high demand for 300GB drives and virtually no demand for old legacy 20MB drives needed for repair. Even trying to buy a 40GB (Gig) drive today is hard. The market has what people call a “sweet spot” – a range of product features and capacity which is the cheapest and easiest to buy. Product builders trying to use components that are better than (or even worse-than) the market sweet-spot have disproportionally higher costs than builders using components in sync with the market sweet-spot.

The same thing happens for Ethernet components – for example buying magnetics rated at 1500v isolation (normal IEEE commercial spec) is very cheap while trying to source magnetics with 2500v isolation can cost an order of magnitude more. So while your company could define a number of electrical improvements for an “Industrial Ethernet” interface, you have to weigh this against the added cost and supply headaches of buying against the grain – of ignoring the gigantic “sweet-spot” for commercial-grade Ethernet components that enable creation of that $9.95 PC Ethernet adapter.

What is Your Manufacturing Sweet-Spot?
Just as the world market has a sweet-spot, so does your own in-house production; just ask your purchasing department. Adding Ethernet is NOT just the cost of adding a few new chips - the NIC, MAC/PHY, magnetics, and RJ45 connector. You may need to upgrade your whole basic hardware design away from a simple 8-bit CPU with 64KByte of memory to a 16 or 32-bit CPU with several MByte of memory. For example, Digi’s basic Device Server platform has a 32-bit CPU, 4MB flash, and 8MB RAM. Few of our products really need this much horsepower, but putting for example 8MB of RAM into all products is cheaper given purchasing logistics and reliability of supply than buying a mix of 2, 4, and 8MB chips. In fact, today we are looking at the cost tradeoff in shifting the basic design from 8MB to 16, 32, or even 64MB. Yes, 16MB (or 64MB) will cost more than 8MB, but given some products need 16MB (or 64MB) there are both tangible and intangible benefits to moving a larger volume of products up the curve to retain supply-chain advantages. This is especially true of FLASH and RAM chips which frequently suffer feast-and-famine availability cycles.

All small companies quickly learn – often the hard way – that during market shortages, it is the small volume purchases that get last delivery. During a chip famine low-volume purchasers will NOT be able to buy sufficient chips at any price to maintain their production. The higher your volume of a part, the lower is your price and perhaps more importantly the more reliable is your supply. So when you start to add Ethernet products and reduce sales of non-Ethernet products, you may find you need to upgrade the CPU design of some your non-Ethernet products to gain or retain reliability of parts supply.

How Robust is Commercial-Grade Ethernet?
So far I have been saying that trying to create special Ethernet hardware for industry may be costly and not very cost-effective. Worse, your average commercial-grade Ethernet is already very robust when compared to RS-232, RS-485 or USB serial. Ethernet uses a transformer-isolated signal with differential pairs, plus has nice, low-level, hardware-supported error detection. Given the high signal frequency, low signal voltage and isolation transformer, trying to add extra surge protection greatly complicates product ground design and weakens the signal, shortening the supported cable length below the 100m length customers have etched within their minds. So trying to boost your Ethernet spec for an industrial design gives questionable gain for the extra cost and lost profits. Plus your customers won’t likely perceive a market differentiation that they are willing to pay for if you say you have better isolation, etc.

A Note on Shielded Ethernet Cables:
Many industrial users start out assuming STP (Shielded Twisted Pair) is better than UTP (Unshielded Twisted Pair) for Ethernet. Oddly enough, STP has proven a bit like ABS brakes in private automobiles; despite lots of hoopla about saving lives when the US government forced ABS brakes into cars, insurance industry records continue to show it has had no measurable impact in real world road deaths. It seems while an expert driver can be helped immensely by ABS, your average idiot or careless driver still reacts to skidding situations in ways ABS brakes cannot fix.

The same appears true for STP cables and Ethernet. I have seem many discussions where industrial users tried STP cables and found the system only works reliably when they lay temporary UTP cables across the weld-shop floor! I suspect the main problem is traditional IT groups have used and measured STP success in terms of preventing Ethernet cable emissions affecting other equipment. This is not the same as using STP to prevent external interference from affecting the Ethernet signal. So ignoring the issues old truisms of a floating shield is worse than no shield and a shield grounded at both ends and creating a ground loop is worse than no shield, it appears that only experts and a very detailed system design results in STP Ethernet working better than UTP. My recommendation is to use optical fiber whenever you really worry about noise interfering with UTP Ethernet.

Vibration and RJ45:
Field tests of RJ45 connectors have shown them very bad in areas of high vibration. This is actually very easy to see for yourself - take any RJ45 connector with pins facing down and wiggle it up and down. What happens? That little finger-catch / lock acts as a pivot point and you are actually scrubbing the gold-flash contacts of the connector against the socket contacts. Metal-against-Metal; quess the result. Tests on industrial robot arms have shown even high-quality gold plated RJ45 connectors self-destruct in months or even weeks. If you expect vibration, better look for alternative connectors - such as any of the many (way too many) IP67 locking designs.

Industry and CAT 5, 5e, and 6:
Another insteresting twist to the commercial evolution of Ethernet is tests of bulk cable shows that CAT 5 is the likely the best for industrial use where the noise rejection properties of the twisted pair (differential signal) is desired. This is because - so I have been told - one of the tradeoffs IEEE allowed for CAT 5e and 6 is to allow less consistancy within the wires of a pair. After all, few Ethernet systems ever see serious external interference, so things which improved speed outweighed things which reduced noise rejection in abnormally high noise conditions. Several large automation companies tried to bring up the idea of a CAT 5i with IEEE which emphasised better noise rejection and special jacket plastic, however ... it appears it went no where. If the big computer, networking, and cable vendors don't see the value, it cannot happen through IEEE.