EDIFACT: The universal message standard
What is EDIFACT?
EDIFACT is the abbreviation for "Electronic Data Interchange for Administration, Commerce and Transport". This is a global set of rules defined by the UN for the inter-company electronic data exchange between two or more business partners via EDI.
It should be noted that EDIFACT is a newer standard for EDI than ANSII X12. X12 is mainly used in America, while EDIFACT is a world-wide standard.
The goal of EDIFACT is the optimization and standardization of the data flow between business partners. By defining uniform segments and elements that describe the information in the electronic file and which are used for a wide variety of document types (such as invoices, purchase orders, delivery notes, etc.) merely by means of a differentiated arrangement, a worldwide standard was created.
However, the EDIFACT standard is very comprehensive and created for almost every business transaction and every industry, subgroups (the so-called subsets) soon emerged. The subset EANCOM was created for the retail sector, which contains the mandatory fields of the EDIFACT standard and the industry-specific optional fields. By creating so-called subsets, the messages are better handled and easier to understand.
The Cost of EDI
The biggest question in EDI is always: What does it cost?
Of course there is no one answer, it depends on who you're dealing with. Are you dealing with a VAN, an integrator, or doing it in-house? Costs vary wildly, and paying more doesn't guarantee you a better system.
VANS - A lower cost and a more expensive cost structure | |
Below is a fairly typical lower cost structure
|
Below is a fairly typical (but expensive) cost structure
|
Yeah, for some companies it's not a problem, but for most small companies paying a VAN is out of the question, it's just too expensive. Fortunately, there are many newer integrators to deal with now, unfortunately their prices are usually lower, but not that much lower.
The cost of VAN's and integrators force most small companies to roll their own EDI systems. While this is lower cost, most small companies don't know enough about EDI to make good decisions on who to hire to get a robust, scalable EDI systems, and they end up with a mishmash of EDI components and no comprehensive system that "just works".
We are not making the claim that our EDI Magic programs and components are the only logical choice, the best choice or the least inexpensive choice if you don't use a VAN or integrator. However, we can't estimate the cost of an open source solution as it depends almost entirely on who you hire to do it for you. We can estimate the costs for EDI Magic though.
We deal with either the Logistics industry or the medical (HIPAA) industry quite often so here's an estimate for a scenario for both of these.
EDI Magic | |
Below is sending a 210 (invoice) and receiving a 997 in reply | Below is sending a 270, and an 837 and receiving 999's, 271's and 835's in reply |
|
|
Total cost (excluding extra programming) $795 | Total cost (excluding extra programming) $2580 |
Why EDI is best for automated data transfer
You need to send some data to another company, and you're wondering how to do it. Generally, the first time it happens you're dealing with a much larger company, and they know how to do it. You're going to use EDI, and that's the end of the investigation phase.
EDI is the most common automated data transfer method for B2B, and for good reason. EDI has been around since the 1950's, it's a well thought out, very standardized file format and file transfer protocols.
Nobody denies that EDI is complex. The EDI specs book for one version of X12 EDI is 1930 pages. However, it's the detail of the specifications that makes EDI the best choice for automated data transfer. In EDI both partners know exactly what format the data will be in, know which elements will be included in the file, and which elements won't be included.
It may not seem obvious at first, but it's the detail and complexity of EDI that has made it successful. It's quite possible when an EDI transaction set is setup properly, it will send thousands or tens of thousands, of files without any errors. When an outgoing EDI file is setup and mapped from a SQL Query (in many cases) to an EDI file it takes developers fluent in EDI, and each data element needs to be mapped. That attention to detail makes it much more likely that data errors will be caught in the setup process and requires a great deal of thought of how to make the incoming data fit the EDI transaction set.
Why is it so important automated files are perfect? Think about it, your trading partner is relying on that data, and normally the data goes right into their system without anyone ever looking at it. Think about sending a company like GM the list of parts you just loaded onto a truck. Then say your data was wrong, and you actually sent 1200 parts but your EDI file said 1000 parts. GM would accept the list without anyone verifying it. If you had a daily shipment and you mistakenly added 200 parts to each list, by the end of the month you would be short thousands of parts, and GM would be paying you for the parts they actually used. Your company could be in big financial trouble quite quickly. Or, let's assume your part numbers were perfect, but your file isn't. The GM EDI engine would reject your file. If you're not reading the response files, you don't know it. At the end of the month, GM doesn't have the information to pay you, and it might take you weeks or months to get it straightened out. In the meantime, your company is out an entire months' worth of parts. It's important that automated files are perfect, and you read the reply files.
Many of your trading partners will have Chargebacks (also known as “expense-offsets”). They are monetary penalties imposed by retailers for non EDI compliance. Retailers impose chargebacks because supplier noncompliance creates disruption and losses for the retailer. Hence, retailers create “expense offset policies” to help them recuperate the losses they experience when suppliers are non-compliant. Retailers usually deduct chargebacks from the check payment to the supplier. Chargebacks are difficult to control because every trading partner has its own chargeback schedule. Suppliers usually get chargebacks because they have issues keeping up with changes from the retailers (vendor manuals and routing instructions are constantly changing).
Each trading partner will have a different chargeback schedule, some retailers charge as much as $50-$100 per EDI mistake or error. Some trading partner’s chargeback schedules contain hundreds of unclear reasons for chargebacks. You should hire an EDI provider that knows every trading partner’s chargeback schedule and EDI requirements.
Fortunately, the EDI specs and your trading partners companion guide lay out exactly what you need to make sure automating data transfer goes smoothly, and hiring an experienced EDI developer to implement it will be money well spent. EDI can be expensive, but once it's setup and running well, your business will run much smoother.
At EDI Shack we take contracts to fix EDI implementations constantly because the original developer didn't quite understand EDI, and implemented a system that sends a file, but doesn't properly keep track of everything. These poor companies have normally spend hundreds or thousands of hours fixing problems that could have been avoided with a better EDI implementation. It's always cheaper to get it done right the first time.
EDI is a System, Not just a File Format
You run a small business, let's say an online store on Shopify. Several of your suppliers have EDI capabilities, and it's obvious to you that some of your processes can be automated if you just knew what EDI is.
EDI is short for "Electronic Document Interface", and it's an accurate description. EDI is a system for exchanging documents between companies in a clear, precise format that eliminates errors and misunderstandings. EDI can be easily automated. However, EDI must be done correctly, or it can be a nightmare.
Imagine you have a small company that sells custom laptops to consumers. Amazon does your shipping for you, so everyday you send dozens of laptops to the Amazon warehouse, and you take orders on your Amazon store and website. Currently, everytime you sell a laptop, you have to enter the information into a form to send it to Amazon. Not only is this system prone to data entry errors, it's time consuming. You decide to automate this, so you hire someone to create an EDI file and send it to Amazon. What you have to realize about EDI is it's automated. Nobody is looking to make sure the data is correct before it's sent to your trading partner. In this case, what if every order gets tripled? What if some orders are never sent? If the orders get tripled, how long can you stay in business if Amazon ships 3 of every laptop you've sold? how many customers will you lose if the order gets lost? When you setup EDI, your system needs to have a mechanism to be 100% sure every order is sent once, and only once. Amazon will of course send back a 997 for every order in an EDI file, but your system has to be capable of matching the
Although an EDI document is simply a text file in EDI format, EDI is MUCH, MUCH MORE than that. EDI is a document management system that specifies how to manage your files, keep track of which documents were received by your trading partner, whether the data was successfully extracted by the EDI system, and whether the data was successfully used. No other document does all that. If you combined a very good document management system and then integrated a file format into it, and then consulted with every major industry on what information they need to share, that's what EDI is.
The EDI file format has wrappers that if properly used, keeps track of not only the file in question, but the files that came before it.
Here's a small checklist of what your EDI system has to do (using the example above). Note that creating the actual EDI file is only a small part of the total.
- Pull the data for all new orders once an hour, or once a day (whatever your timeline is)
- make sure to ONLY pull the information for paid orders
- check for missing information. EDI should NEVER send incomplete information
- notify staff if there are orders completed, but still missing information
- using the data pulled, format it into EDI formats
- Compose the EDI file
- Store the EDI file in long term storage
- put a copy of the EDI file in a directory you will send from
- Start the sending system so the file gets sent to your trading partner
- Verify the file was sent properly, AND received by the trading partner
- Check to see if there are any reply files that need to be read
- for every reply file, match the data to your data and update the entries
- Close down properly without any memory leaks
Note that composing the EDI file is about halfway in the total process and there are a lot of things to be done BEFORE and AFTER the file is actually composed
EDI Terminology
Addenda – Refers to a version of the HIPAA mandated transaction sets that corrects identified implementation issues noted in the original TR3.
ASC X12N/005010X222– The HIPAA mandated (ANSI) ASC X12N Professional Health Care Claim transaction format.
ASC X12N/005010X222A1 – The Type 1 Errata modifications mandated for use with the ASC X12N/005010X222 837 Professional Health Care Claim transaction format.
BCBSM or FEP Supplemental – BCBSM or FEP is being billed as the secondary payer and the primary payer is original Medicare or any type of Medicare Advantage or Medicare Advantage HMO.
BCBSA – An acronym for Blue Cross Blue Shield Association
BCC – An acronym for Blue Cross Complete of Michigan, a Medicaid managed care plan
BCN – An acronym for Blue Care Network
BlueExchange – A BCBSA process through which non-claim HIPAA transactions for members from all other Blue Cross and/or Blue Shield plans that are governed by the BCBSA can be accepted by a local host plan (the plan that delivers the benefits to a member) and routed to the home plan (the plan that covers the member) for processing.
Data Segment – Corresponds to a record in data processing terminology. Consists of logically related data elements in a defined sequence (defined by X12N). Each segment begins with a segment identifier, which is not a data element and one or more related data elements, which are preceded by a data element separator. Each segment ends with a segment terminator.
Data Element – Corresponds to a field in data processing terminology. Assigned unique reference number. Each element has a name, description, type, minimum length and maximum length. The length of an element is the number of character positions used, except as noted for numeric, decimal and binary elements. Data element types are defined in Appendices B of the TR3.
Delimiter – A character used to separate two data elements (or sub-elements) or to end a segment. They are specified in the interchange header segment (ISA). Once specified in the ISA, they should not be used in the data elsewhere other than as a separator or terminator.
EDI – An acronym for Electronic Data Interchange.
Electronic Data Interchange – The application-to-application transfer of key business information transacted in a standard format using a computer-to-computer communications link. There are typically 6 components used in order to do EDI. They are: an EDI file, a trading partner, an application file/form, translator (mapper), communications and value-added network or value-added service provider.
FEP – Federal Employee Program
Interface – The point at which two systems connect to pass data
Loops – Loops are groups of semantically related segments. Data segment loops may be unbounded or bounded.
Medicare Beneficiary Identifier (MBI) – an eleven-character alpha numeric identification number issued by the Center for Medicare Services, which replaces the Medicare Health Insurance Claim Number (HICN).
NASCO – The National Account Service Company connects several Blue Cross and Blue Shield plans across the country through a common automated system to administer health benefit programs.
Routing – Separation of data based on specific criteria for subsequent transfer to an internal or external system.
Technical Reports Type 3 (TR3s) – Documents that provide standardized data requirements and content as the specifications for consistent implementation of a standard transaction set. The Washington Publishing Company publishes HIPAA TR3s on their web site: http://store.x12.org/store/
Trading partners – Companies/Entities that exchange electronic data files. Agreements are sometimes made between the partners to define the parameters of the data exchange and simplify the implementation process.
Transaction Set – A transaction set is considered one business document which is composed of a transaction set header control segment, one or more data segments, and a transaction set trailer control segment.
X12 - An Accredited Standards Committee commissioned by the American National Standards Institute to develop standards for Electronic Data Interchange
X12N – An Accredited Standards Committee commissioned by the American National Standards Institute to develop standards for Electronic Data Interchange. While X12 indicates EDI, the N identifies the Insurance Subcommittee that is responsible for developing EDI standards for the insurance industry. There is a special health care task group within this subcommittee responsible for the development of health care insurance transactions.
Online EDI Resources
Well, of course you're here, so you know about EDI Shack, but there are many online EDI resources. This list is not complete, but hopefully it is useful.
EDI VALIDATORS
https://www.edivalidation.com/
Libraries / Frameworks
Python library for parsing ANSI ASC C12 files BadX12