Electronic data interchange

Electronic data interchange (EDI) is the concept of businesses electronically communicating information that was traditionally communicated on paper, such as purchase orders, advance ship notices, and invoices. Technical standards for EDI exist to facilitate parties transacting such instruments without having to make special arrangements.

EDI has existed at least since the early 70s, and there are many EDI standards (including X12, EDIFACT, ODETTE, etc.), some of which address the needs of specific industries or regions. It also refers specifically to a family of standards. In 1996, the National Institute of Standards and Technology defined electronic data interchange as "the computer-to-computer interchange of a standardized format for data exchange. EDI implies a sequence of messages between two parties, either of whom may serve as originator or recipient. The formatted data representing the documents may be transmitted from originator to recipient via telecommunications or physically transported on electronic storage media." It distinguished mere electronic communication or data exchange, specifying that "in EDI, the usual processing of received messages is by computer only. Human intervention in the processing of a received message is typically intended only for error conditions, for quality review, and for special situations. For example, the transmission of binary or textual data is not EDI as defined here unless the data are treated as one or more data elements of an EDI message and are not normally intended for human interpretation as part of online data processing." In short, EDI can be defined as the transfer of structured data, by agreed message standards, from one computer system to another without human intervention.

History

Like many other early information technologies, EDI was inspired by developments in military logistics. The complexity of the 1948 Berlin airlift required the development of concepts and methods to exchange, sometimes over a 300 baud teletype modem, vast quantities of data and information about transported goods. These initial concepts later shaped the first TDCC (Transportation Data Coordinating Committee) standards in the US. Among the first integrated systems using EDI were Freight Control Systems. One such real-time system was the London Airport Cargo EDP Scheme (LACES) at Heathrow Airport, London, UK, in 1971. Implementing the direct trader input (DTI) method, it allowed forwarding agents to enter information directly into the customs processing system, reducing the time for clearance. The increase of maritime traffic and problems at customs similar to those experienced at Heathrow Airport led to the implementation of DTI systems in individual ports or groups of ports in the 1980s.

Standards

EDI provides a technical basis for automated commercial "conversations" between two entities, either internal or external. The term EDI encompasses the entire electronic data interchange process, including the transmission, message flow, document format, and software used to interpret the documents. However, EDI standards describe the rigorous format of electronic documents, and the EDI standards were designed, initially in the automotive industry, to be independent of communication and software technologies.

EDI documents generally contain the same information that would normally be found in a paper document used for the same organizational function. For example, an EDI 940 ship-from-warehouse order is used by a manufacturer to tell a warehouse to ship a product to a retailer. It typically has a 'ship-to' address, a 'bill-to' address, and a list of product numbers (usually a UPC) and quantities. Another example is the set of messages between sellers and buyers, such as request for quotation (RFQ), bid in response to RFQ, purchase order, purchase order acknowledgement, shipping notice, receiving advice, invoice, and payment advice. However, EDI is not confined to just business data related to trade but encompasses all fields such as medicine (e.g., patient records and laboratory results), transport (e.g., container and modal information), engineering and construction, etc. In some cases, EDI will be used to create a new business information flow (that was not a paper flow before). This is the case in the Advanced Shipment Notification (ASN) which was designed to inform the receiver of a shipment, the goods to be received and how the goods are packaged. This is further complemented with the shipment's use of the shipping labels containing a GS1-128 barcode referencing the shipment's tracking number.

Some major sets of EDI standards:

• The UN-recommended UN/EDIFACT is the only international standard and is predominant outside of North America.
• The US standard ANSI ASC X12 (X12) is predominant in North America.
• GS1 EDI set of standards developed the GS1, predominant in global supply chain.
• The TRADACOMS standard developed by the ANA (Article Number Association, now known as GS1 UK) is predominant in the UK retail industry.
• The ODETTE standard used within the European automotive industry.
• The VDA standard used within the European automotive industry, mainly in Germany.
• HL7, a semantic interoperability standard used for healthcare data.
• HIPAA, The Health Insurance Portability and Accountability ACT (HIPAA), requires millions of healthcare entities who electronically transmit data to use EDI in a standard HIPAA format.
• IATA Cargo-IMP, IATA Cargo-IMP stands for International Air Transport Association Cargo Interchange Message Procedures. It is an EDI standard based on EDIFACT created to automate and standardize data exchange between airlines and other parties.
• NCPDP Script, SCRIPT is a standard developed and maintained by the National Council for Prescription Drug Programs (NCPDP). The standard defines documents for electronic transmission of medical prescriptions in the United States.
• The NCPDP Telecommunications standard includes transactions for eligibility verification, claim and service billing, predetermination of benefits, prior authorization, and information reporting, and is used primarily in the United States.
• Edig@s (EDIGAS) is a standard dealing with commerce, transport (via pipeline or container) and storage of gas.

Many of these standards first appeared in the early to mid-1980s. The standards prescribe the formats, character sets, and data elements used in the exchange of business documents and forms. The complete X12 Document List includes all major business documents, including purchase orders and invoices.

The EDI standard prescribes mandatory and optional information for a particular document and gives the rules for the structure of the document. The standards are like building codes. Just as two kitchens can be built "to code" but look completely different, two EDI documents can follow the same standard and contain different sets of information. For example, a food company may indicate a product's expiration date while a clothing manufacturer would choose to send colour and size information.

Transmission protocols

EDI can be transmitted using any methodology agreed to by the sender and recipient, but as more trading partners began using the Internet for transmission, standardized protocols have emerged.

This includes various technologies such as:

• mModem (asynchronous and synchronous)
• FTP, SFTP and FTPS
• Email
• HTTP/HTTPS
• AS1
• AS2
• AS4
• OFTP (and OFTP2)
• Mobile EDI

When some people compared the synchronous protocol 2400 bit/s modems, CLEO devices, and value-added networks used to transmit EDI documents to transmitting via the Internet, they equated the non-Internet technologies with EDI and predicted erroneously that EDI itself would be replaced along with the non-Internet technologies. In most cases, these non-internet transmission methods are simply being replaced by Internet protocols, such as FTP, HTTP, telnet, and e-mail, but the EDI documents themselves still remain.

In 2002, the IETF published RFC 3335, offering a standardized, secure method of transferring EDI data via e-mail. On July 12, 2005, an IETF working group ratified RFC4130 for MIME-based HTTP EDIINT (a.k.a. AS2) transfers, and the IETF has prepared a similar RFC for FTP transfers (a.k.a. AS3). EDI via web services (a.k.a. AS4) has also been standardized by the OASIS standards body. While some EDI transmission has moved to these newer protocols, the providers of value-added networks remain active.

Internet

As more organizations connected to the Internet, eventually most or all EDI was pushed onto it. Initially, this was through ad hoc conventions, such as unencrypted FTP of ASCII text files to a certain folder on a certain host, permitted only from certain IP addresses. However, the IETF has published several informational documents (the "Applicability Statements"; see below under Protocols) describing ways to use standard internet protocols for EDI.

As of 2002, Walmart has pushed AS2 for EDI. Because of its significant presence in the global supply chain, AS2 has become a commonly adopted approach for EDI.

Specifications

Organizations that send or receive documents from each other are referred to as "trading partners" in EDI terminology. The trading partners agree on the specific information to be transmitted and how it should be used. This is done in human-readable specifications (also called Message Implementation Guidelines). While the standards are analogous to building codes, the specifications are analogous to blueprints. (The specification may also be called a "mapping," but the term mapping is typically reserved for specific machine-readable instructions given to the translation software.) Larger trading "hubs" have existing Message Implementation Guidelines which mirror their business processes for processing EDI and they are usually unwilling to modify their EDI business practices to meet the needs of their trading partners. Often in a large company, these EDI guidelines will be written to be generic enough to be used by different branches or divisions and therefore will contain information not needed for a particular business document exchange. For other large companies, they may create separate EDI guidelines for each branch/division.

Transmission: Direct EDI and VANs

Trading partners are free to use any method for the transmission of documents (as described above in the Transmission protocols section). Further, they can either interact directly or through an intermediary.

Direct EDI: peer-to-peer

Trading partners can connect directly to each other. For example, an automotive manufacturer might maintain a modem-pool that all of its hundreds of suppliers are required to dial into to perform EDI. However, if a supplier does business with several manufacturers, it may need to acquire a different modem (or VPN device, etc.) and different software for each one.

As EDI and web technology have evolved, new EDI software technologies have emerged to facilitate direct (also known as point-to-point) EDI between trading partners. Modern EDI software can facilitate exchanges using any number of different file transmission protocols and EDI document standards, reducing costs and barriers to entry.

Value-added networks

To address the limitations in peer-to-peer adoption of EDI, VANs (value-added networks) were established decades ago. A VAN acts as a regional post office. It receives transactions, examines the 'from' and the 'to' information, and routes the transaction to the final recipient. VANs may provide a number of additional services, e.g. retransmitting documents, providing third party audit information, acting as a gateway for different transmission methods, and handling telecommunications support. Because of these and other services VANs provide, businesses frequently use a VAN even when both trading partners are using Internet-based protocols. Healthcare clearinghouses perform many of the same functions as a VAN, but have additional legal restrictions.

VANs may be operated by various entities:

• telecommunication companies;
• industry group consortia;
• a large company interacting with its suppliers/vendors;
• managed services providers.

Costs, trade-offs and implementation

It is important to note that there are key trade-offs between VANs and Direct EDI, and in many instances, organizations exchanging EDI documents can in fact use both in concert, for different aspects of their EDI implementations. For example, in the U.S., the majority of EDI document exchanges use AS2, so a direct EDI setup for AS2 may make sense for a U.S.-based organization. But adding OFTP2 capabilities to communicate with a European partner may be difficult, so a VAN might make sense to handle those specific transactions, while direct EDI is used for the AS2 transactions.

In many ways, a VAN acts as a service provider, simplifying much of the setup for organizations looking to initiate EDI. Due to the fact that many organizations first starting out with EDI often do so to meet a customer or partner requirement and therefore lack in-house EDI expertise, a VAN can be a valuable asset.

However, VANs may come with high costs. VANs typically charge a per-document or even per-line-item transaction fee to process EDI transactions as a service on behalf of their customers. This is the predominant reason why many organizations also implement an EDI software solution or eventually migrate to one for some or all of their EDI.

On the other hand, implementing EDI software can be a challenging process, depending on the complexity of the use case, technologies involved and availability of EDI expertise. In addition, there are ongoing maintenance requirements and updates to consider. For example, EDI mapping is one of the most challenging EDI management tasks. Companies must develop and maintain EDI maps for each of their trading partners (and sometimes multiple EDI maps for each trading partner based on their order fulfilment requirements).

Interpreting data

EDI translation software provides the interface between internal systems and the EDI format sent/received. For an "inbound" document, the EDI solution will receive the file (either via a value-added network or directly using protocols such as FTP or AS2), take the received EDI file (commonly referred to as an "envelope"), and validate that the trading partner who is sending the file is a valid trading partner, that the structure of the file meets the EDI standards, and that the individual fields of information conform to the agreed-upon standards. Typically, the translator will either create a file of either fixed length, variable length or XML tagged format or "print" the received EDI document (for non-integrated EDI environments). The next step is to convert/transform the file that the translator creates into a format that can be imported into a company's back-end business systems, applications or ERP. This can be accomplished by using a custom program, an integrated proprietary "mapper" or an integrated standards-based graphical "mapper," using a standard data transformation language such as XSLT. The final step is to import the transformed file (or database) into the company's back-end system.

For an "outbound" document, the process for integrated EDI is to export a file (or read a database) from a company's information systems and transform the file to the appropriate format for the translator. The translation software will then "validate" the EDI file sent to ensure that it meets the standard agreed upon by the trading partners, convert the file into "EDI" format (adding the appropriate identifiers and control structures) and send the file to the trading partner (using the appropriate communications protocol).

Another critical component of any EDI translation software is a complete "audit" of all the steps to move business documents between trading partners. The audit ensures that any transaction (which in reality is a business document) can be tracked to ensure that they are not lost. In the case of a retailer sending a Purchase Order to a supplier, if the Purchase Order is "lost" anywhere in the business process, the effect is devastating to both businesses. To the supplier, they do not fulfil the order as they have not received it thereby losing business and damaging the business relationship with their retail client. For the retailer, they have a stock outage and the effect is lost sales, reduced customer service and ultimately lower profits.

In EDI terminology, "inbound" and "outbound" refer to the direction of transmission of an EDI document in relation to a particular system, not the direction of merchandise, money or other things represented by the document. For example, an EDI document that tells a warehouse to perform an outbound shipment is an inbound document in relation to the warehouse computer system. It is an outbound document in relation to the manufacturer or dealer that transmitted the document.

Advantages over paper systems

EDI and other similar technologies save the company money by providing an alternative to or replacing, information flows that require a great deal of human interaction and paper documents. Even when paper documents are maintained in parallel with EDI exchange, e.g. printed shipping manifests, electronic exchange and the use of data from that exchange reduces the handling costs of sorting, distributing, organizing, and searching paper documents. EDI and similar technologies allow a company to take advantage of the benefits of storing and manipulating data electronically without the cost of manual entry. Another advantage of EDI is the opportunity to reduce or eliminate manual data entry errors, such as shipping and billing errors, because EDI eliminates the need to re-key documents on the destination side. One very important advantage of EDI over paper documents is the speed at which the trading partner receives and incorporates the information into their system greatly reducing cycle times. For this reason, EDI can be an important component of just-in-time production systems.

According to the 2008 Aberdeen report "A Comparison of Supplier Enablement around the World", only 34% of purchase orders are transmitted electronically in North America. In EMEA, 36% of orders are transmitted electronically and in APAC, 41% of orders are transmitted electronically. They also report that the average paper requisition to order costs a company $37.45 in North America, $42.90 in EMEA and $23.90 in APAC. With an EDI requisition to order, costs are reduced to $23.83 in North America, $34.05 in EMEA and $14.78 in APAC.

Barriers to implementation

There are a few barriers to adopting electronic data interchange. One of the most significant barriers is the accompanying business process change. Existing business processes built around paper handling may not be suited for EDI and would require changes to accommodate automated processing of business documents. For example, a business may receive the bulk of their goods by 1 or 2-day shipping and all of their invoices by mail. The existing process may, therefore, assume that goods are typically received before the invoice. With EDI, the invoice will typically be sent when the goods ship and will, therefore, require a process that handles large numbers of invoices whose corresponding goods have not yet been received.

Another significant barrier is the cost in time and money in the initial setup. The preliminary expenses and time that arise from the implementation, customization and training can be costly. It is important to select the correct level of integration to match the business requirement. For a business with relatively few transactions with EDI-based partners, it may make sense for businesses to implement inexpensive "rip and read" solutions, where the EDI format is printed out in human-readable form, and people — rather than computers — respond to the transaction. Another alternative is outsourced EDI solutions provided by EDI "Service Bureaus". For other businesses, the implementation of an integrated EDI solution may be necessary as increases in trading volumes brought on by EDI force them to re-implement their order processing business processes.

The key hindrance to a successful implementation of EDI is the perception many businesses have of the nature of EDI. Many view EDI from the technical perspective that EDI is a data format; it would be more accurate to take the business view that EDI is a system for exchanging business documents with external entities, and integrating the data from those documents into the company's internal systems. Successful implementations of EDI take into account the effect externally generated information will have on their internal systems and validate the business information received. For example, allowing a supplier to update a retailer's accounts payable system without appropriate checks and balances would put the company at significant risk. Businesses new to the implementation of EDI must understand the underlying business process and apply proper judgment.

Acknowledgement

There are several mechanisms used in EDI for acknowledgement, i.e. notifying the sender that an incoming transaction was received and handled by the recipient:

• Communication Status – Indicate the transmission completed
• MDN (Message Disposition Notification) – In AS2 only, indicate the message is readable
• Functional Acknowledgement – typically "997" in ANSI, or "CONTRL" in EDIFACT, which indicate the message content is verified against its template, and tell if the transaction is posted to the receiver's electronic system.
• Business Level Acknowledgement – the final indicator shows if the transaction is accepted by the receiver or not.

See also

• Expense and Cost Recovery System (ECRS)
• Extract, transform, load (ETL)
• Legal Electronic Data Exchange Standard (LEDES)
• Maritime data standards

Protocols

• HTTP/HTTPS
• POP3/SMTP
• OFTP/OFTP2
• SOAP
• WebDAV
• X.400
• EDIINT working group:
  • EDIINT AS1 (extension to mail transport)
  • EDIINT AS2 (based on HTTP transport)
  • EDIINT AS3 (based on FTP transport)
  • EDIINT AS4 (based on WebServices)

Formats

• ANSI X.12
  • X12
  • X12 Document List
  • X12 EDIFACT Mapping
• XML
  • cXML
  • xCBL
  • ebXML
  • railML
  • RosettaNet
  • UBL
  • UNIDOC
• Tradacoms
• EDIFACT
• Cefic – Chemical
• GS1 EANCOM – Retail
• EDIBDB – Construction
• EDIFICE – High Tech Industry
• EDIFURN – Furniture
• EDILEKTRO – Electro
• EDILIBE – Books
• EDITEC – Sanitary
• EDITEX – Fashion
• EDIFOR/EDITRANS – Transports & Logistics
• EDIWHEEL – Wheels & Tires
• ETIS – Telecommunication
• STAR – Standards for Technology in Automotive Retail
• CXIO (Commerce Exchange Information Objects) (external link)
• SPEC2000 (Airline Industry) (external link Archived 2011-07-16 at the Wayback Machine)

Fixed-length formats

• EURITMO

Separator formats

• CSV/TSV/DSV

Further reading

• Gengeswari, K. and Abu Bakar Abdul Hamid (2010). "Integration of electronic data interchange: a review", Jurnal Kemanusiaan, ISSN 1675-1930
• Notto, Ralph W. "Challenge and Consequence: Forcing Change to ECommerce", Fenestra Books. 2005, ISBN 158736414X

External links

• "E-Procurement — Electronic Data Integration Comes of Age" Archived 2021-05-14 at the Wayback Machine – Article from Finance Director Europe Journal
• EDI Protocols – An overview of the various protocols and formats used in EDI networks
• EDI Standards – An overview of the various EDI file format standards available.