03Sheet 03.08 · Tools

Conformance

make conformance: the same vectors through JavaScript and Go SDK builds before and after a fix, plus rs-stellar-xdr. Known bugs fail on the build before their fix and pass after.

make conformance     # conformance/run.sh

The SDK builds

SDKBuilds
js-stellar-sdk17.2.0, 17.2.1, 17.3.0
go-stellar-sdkv0.7.1, v0.7.2, v0.9.0
rs-stellar-xdr30.0.0

Each build runs as a small testee that speaks line-delimited JSON. The orchestrator sends it every vector, with a 10-second timeout each, and judges the answer. The verdicts are equals, roundtrip (bytes, plus JSON where both sides have it), error, error_or_equals or error_or_count. An operation a testee doesn't support is n/a.

The recorded matrix

The run compares its result with the committed matrix, conformance/recorded/matrix-sightings.md. Any difference fails it. This is that matrix:

Vectorjs@17.2.0js@17.2.1js@17.3.0rust@30.0.0go@v0.7.1go@v0.7.2go@v0.9.0
asset-order/xdr-vs-strkey/compareFAILpasspasspassFAILpasspass
asset-order/xdr-vs-strkey/pool-share-change-trust/rawpasspasspasspasspasspasspass
asset-order/xdr-vs-strkey/pool-share-change-trust/helperFAILpasspassn/an/an/an/a
scmap/colliding-keys/rawpasspasspasspasspasspasspass
scmap/colliding-keys/to-nativeFAILFAILpassn/an/an/an/a
muxed/id-canonical/hex-stringFAILFAILpassn/an/an/an/a
muxed/id-canonical/empty-stringFAILFAILpassn/an/an/an/a
offer-id/above-2^53-as-numberFAILFAILpassn/an/an/an/a
offer-id/negativeFAILFAILpassn/an/an/an/a
offer-id/empty-stringFAILFAILpassn/an/an/an/a

What every FAIL says

  • js@17.2.0 on asset-order/xdr-vs-strkey/compare: wrong value: 1
  • go@v0.7.1 on asset-order/xdr-vs-strkey/compare: wrong value: 1
  • js@17.2.0 on asset-order/xdr-vs-strkey/pool-share-change-trust/helper: api error: Assets are not in lexicographic order
  • js@17.2.0 on scmap/colliding-keys/to-native: lossy: 1 of 2 entries: {"$object":[["1",20]]}
  • js@17.2.1 on scmap/colliding-keys/to-native: lossy: 1 of 2 entries: {"$object":[["1",20]]}
  • js@17.2.0 on muxed/id-canonical/hex-string: wrong value: {"address":"MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  • js@17.2.1 on muxed/id-canonical/hex-string: wrong value: {"address":"MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  • js@17.2.0 on muxed/id-canonical/empty-string: accepted: {"address":"MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  • js@17.2.1 on muxed/id-canonical/empty-string: accepted: {"address":"MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  • js@17.2.0 on offer-id/above-2^53-as-number: wrong value: {"offerId":"9007199254740992"}
  • js@17.2.1 on offer-id/above-2^53-as-number: wrong value: {"offerId":"9007199254740992"}
  • js@17.2.0 on offer-id/negative: accepted: {"offerId":"-1"}
  • js@17.2.1 on offer-id/negative: accepted: {"offerId":"-1"}
  • js@17.2.0 on offer-id/empty-string: accepted: {"offerId":"0"}
  • js@17.2.1 on offer-id/empty-string: accepted: {"offerId":"0"}

So, in this matrix:

  • the #1766 fix appears at js 17.2.1 and go v0.7.2;
  • the #1785 and #1789 fixes appear at js 17.3.0.

Vectors, not entries

The runner uses its own 10 vectors in conformance/vectors/sightings.json, three for #1766, two for #1789 and five for #1785. It does not read the corpus entries or run their SDK checks yet.

On this page