Dataset
The same shipment record read by gateway — which ports carry which commodities, on which lanes, and what that implies for transit time and cost.
Two suppliers quoting the same ex-works price are not offering the same deal if one ships from a port with weekly direct sailings and the other transships twice. The port pair on a record is the closest thing to a free freight quotation you will get.
Direct versus transhipped changes the lead time materially, and the manifest shows which it was.
Lane volume drives rate. High-volume lanes are cheaper and more reliable.
Ports concentrated in one commodity behave differently in a peak.
For India, the ICD a consignment clears at determines the road or rail leg.
Indian trade splits across the western ports (Nhava Sheva, Mundra, Kandla, Hazira, Pipavav), the southern and eastern ports (Chennai, Tuticorin, Cochin, Visakhapatnam, Paradip, Kolkata) and a large inland container depot network feeding them. Where a consignment clears tells you roughly where the counterparty is.
No single dataset describes a shipment completely. A customs declaration is precise about classification and value because duty is assessed on it. A carrier manifest is precise about parties, vessel and container because carriage depends on it. Statistical aggregates are reconciled and comparable across countries but carry no counterparty at all. Each was created for a different purpose, and each is authoritative about the fields that purpose required.
The practical implication is that port wise trade data answers some questions definitively and others not at all, and the skill is in knowing which is which before you build a decision on it. Using a dataset outside the questions it can answer is the most common way that a technically correct analysis reaches a wrong conclusion.
| Dataset | Authoritative about | Silent on |
|---|---|---|
| Customs declarations | Classification, declared value, duty basis, quantity | Vessel, container, carrier; counterparty in some jurisdictions |
| Bills of lading | Shipper, consignee, vessel, container, port pair | Tariff line and assessed value |
| Statistical aggregates | Comparable country totals by product and partner | Any individual company or consignment |
| Company profiles | One party's history assembled across filings | Anything that depends on unresolved name variants being correct |
Three properties determine whether a dataset can support the decision you want to make. Coverage is which markets and which fields are actually present — and it is never uniform, because it depends on what each authority publishes rather than on what anyone would prefer. Lag is how long after the event the record appears, typically thirty to sixty days. Revision is the fact that published periods are restated as late filings and corrections arrive.
The most common analytical error follows directly from the second and third. The final one or two periods of any series are incomplete and will fill in after you look at them. Reading that as a decline produces confident conclusions about buyers who have not stopped buying and markets that have not contracted. Drop the tail before you read a trend, and expect published figures to move slightly between pulls.
Anyone integrating trade data discovers the same four problems, usually in the same order. Classification changes between HS editions, so a code has to be stored with its edition or a time series will break silently at the revision boundary. Statutory units differ by tariff line and by country, so quantities have to be normalised on ingest rather than at read time. Values are declared on different bases — free on board for exports, cost-insurance-freight for imports — so they are not directly comparable. And company names vary across filings, so any aggregation by party depends on entity resolution that someone has to have done well.
Decide this before you integrate anything. Reporting and market sizing want aggregates; sales and sourcing workflows want individual records. The two have very different volume profiles, refresh needs and storage implications, and building for one when you needed the other is an expensive discovery.
Four questions catch most problems. Which period does this cover, and is that period complete? What basis is the value on, and is the comparison like for like? What unit is the quantity in, and is it the same unit across the rows being compared? And has value moved because volume moved, or because price moved — because those lead to opposite decisions and value alone cannot distinguish them.
None of that is difficult, but it is the difference between an analysis that survives scrutiny and one that collapses the first time somebody knowledgeable asks how the number was constructed. Every table on this site carries the period and the source it came from for exactly that reason.
Trade data is sold by a lot of people, and the differences between offerings are not usually visible in a demo. The questions below are the ones that separate a dataset you can build on from one that will quietly embarrass you in six months, and none of them are unreasonable to ask before committing.
| Question | What a good answer sounds like |
|---|---|
| Which markets, and which fields in each? | A specific list, with the gaps named rather than glossed over |
| What is the lag, market by market? | A number per market, not a single marketing figure for all of them |
| How are blanks handled? | Left blank when the source did not publish, never estimated and never silently filled |
| How is entity resolution done? | An explained method, with the variants visible so over-merging can be spotted |
| Are historical periods revised? | Yes, with restatements passed through rather than frozen at first publication |
| What happens at an HS revision? | Codes stored with their edition, so time series do not break at the boundary |
Most of the practical value comes from a small number of habits rather than from any sophisticated technique. Save the queries you run repeatedly rather than rebuilding them. Keep the period stamp attached to every figure you export, because a number without a period is not evidence. Normalise units once, on the way in, rather than repeatedly at the point of use. And separate volume from value in every chart you make, because the two answer different questions and combining them hides both.
When something looks surprising, check the boring explanations first. An unexpected spike is more often a bulk parcel, a reclassification or a late batch of filings than a market event. An unexpected collapse at the end of a series is almost always reporting lag. Genuine surprises do exist and they are valuable, but they are outnumbered by artefacts, and the discipline of ruling those out first is what makes the genuine ones credible when you report them.
Every figure you circulate should be able to answer three questions: which source, which period, and on what basis the value was declared. An analysis that cannot answer those will not survive its first serious review, however good the underlying data was.
On the source authority's own release cycle rather than on ours — monthly for most detailed markets, and a 45 to 60 day cycle for a few jurisdictions. The most recent one or two periods are always still filling in, so exclude them when you are reading a trend.
Coverage differs by dataset and by market, because it depends on what each authority publishes. Detailed shipment-level coverage is strongest across India, the United States, South East Asia and Latin America, with statistical coverage everywhere else. We name the gaps rather than papering over them.
No. Where a source did not publish a field, it is left blank. An estimated counterparty or an inferred value is worse than an acknowledged gap, because it looks identical to a real one in every downstream calculation.
Yes. Search results export to Excel and CSV, scheduled extracts deliver a fixed query on a schedule, bulk files load history into a warehouse, and a query endpoint serves records to an application on demand.
Store the code together with the Harmonized System edition it belongs to. The nomenclature is revised periodically and codes move, so a series keyed on the code alone will break silently at the revision boundary and look like a market event.
No. The country and chapter tables are official aggregates reported to UN Comtrade — country-level, annual and reconciled across reporters. This dataset is the underlying transactional record, which names parties and describes individual consignments.
That is usually where the value compounds — a declaration joined to a manifest gives you classification and value alongside the named parties and the lane. Plan for entity resolution on the join, because company names will not match cleanly across sources.
Name the market and the HS code and we will send live records in this shape.