Prepare for prov 3.3.0¶
prov 3.3.0 rejects some input that 3.2.x read, and reads a few other inputs
differently. This guide lists each error and warning, the input that causes it and what
to write instead. On prov 3.2.3, the same page at
https://prov.readthedocs.io/en/3.2.3/howto/prepare-for-3-3.html describes the warnings
that release raises for most of these inputs, and how to turn them into errors to find
them before upgrading.
Read the message¶
A message from a reader starts with the position of the input, which is the line and
column in PROV-N, or the record identifier and attribute name in PROV-JSON. A language
tag, a local part, a namespace prefix, a namespace IRI, a Python value, a qualified-name
value or a time offset beyond ±14:00 is rejected by the model, and the model’s sentence
names the value. A reader’s error names the record (the subject in PROV-O), or the
declaration. It carries that sentence, and its __cause__ is the model’s error. Code that
builds the value gets the model’s error on its own. A message from unified() names the
identifier of the records it cannot merge.
With prov.read() and no format=, a PROV-JSON, PROV-JSONLD or PROV-N input that the
reader rejects raises ProvFormatNotDetected, a TypeError, whose __cause__ is the
reader’s error with the rule and its position. That holds for a rejection by the model
too, since the reader wraps it. A PROV-XML or PROV-O input chains nothing, because its
first character does not name its reader. Pass format= to see the rule on its own.
Read and write documents shows the except clauses to write.
PROV-N declarations¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
One |
|
|
|
A prefix that starts with a letter, such as |
|
|
|
Add |
|
|
|
The |
|
|
|
No declaration. Both prefixes are predeclared. |
Each is a ProvNSyntaxError at the declaration or, for an undeclared xsi: name, at
the name. In 3.2.3 the first two rows and the two strict-profile rows warn with the same
text; an undeclared xsi: name warns prefix 'xsi' is not declared. prov 3.2.2 and
earlier wrote xsi: names without a prefix xsi line. Reading such a file and writing
it again with prov 3.2.3 adds the declaration, so do that before upgrading.
PROV-N whitespace¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A space, tab, CR or LF. |
Each is a ProvNSyntaxError at the character, and the same error names any other Unicode
whitespace character between tokens, such as U+000C (form feed) or U+2028 (line
separator). U+1680 is a name character, so entity( followed by U+1680 and ex:e) fails
at the name instead. Whitespace inside a string, an IRI or a comment is unchanged.
PROV-N statements¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The value as the argument, |
|
|
|
Every statement before the first bundle. |
|
|
|
|
|
|
|
|
Each is a ProvNSyntaxError at the attribute name, the statement, the identifier or the
[. The identifier and attribute-block errors name whichever of alternateOf,
specializationOf and hadMember carries one. The default and lenient profiles
accept the three strict-profile constructs in this table. Under lenient, the first row
is a skipped statement with a ProvWarning, like any statement that fails to parse. In
3.2.3 a lenient read kept that statement and warned, so an error filter on
ProvFutureRejectionWarning raised then and does not now.
The strict writer drops what the strict reader rejects:
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The record without them, or |
Times¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The same instant with an offset from -14:00 to +14:00, such as |
|
|
|
At most six fractional digits, |
The error is a ProvValueError where the time is asserted, from code, for every time of a
statement and for a Python datetime as the value of any attribute. A reader raises its
format’s error with it as the cause. 3.2.3 warned only for a time of a statement. A typed
xsd:dateTime value of an ordinary attribute with such an offset,
[ex:t="2011-11-16T16:05:33+15:00" %% xsd:dateTime], is not an error. 3.3.0 keeps it as a
Literal with its text, where 3.2.3 read it as a datetime and warned. A PROV-N read
raises a ProvNSyntaxError at the statement, and the lenient profile skips the
statement with a ProvWarning.
A time string must be in the xsd:dateTime lexical form, YYYY-MM-DDThh:mm:ss, with
optional fractional seconds and Z or an hh:mm offset. 3.2.3 read the other ISO 8601
forms Python reads and did not warn. From 3.3.0 2012-12-03T21:08Z, 20121203T210816Z
and 2012-W01-1T21:08:16Z are not times.
Local parts¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A local part without the character, such as |
The error is a ProvExceptionInvalidQualifiedName when the name is built, from code or
from a pickle written by an earlier release; its message is
Invalid Qualified Name: ex:a b, followed by the text in the table. A reader raises its
format’s error with it as the cause. A PROV-N read raises a ProvNSyntaxError at the
statement, and the lenient profile skips the statement with a ProvWarning. A local
part cannot hold a space, a control character or another character that PROV-N can write
only by percent-encoding it, such as " or |. Nor can it hold a % that two
hexadecimal digits do not follow, or start with U+00B7 (middle dot), U+203F (undertie),
U+2040 (character tie) or a combining mark from U+0300 to U+036F. 3.2.3 percent-encoded
such a character when it wrote PROV-N, so a PROV-N reader recovered a different IRI. The
characters = ' ( ) , : ; [ ] are written with a backslash instead, and are accepted.
Namespace prefixes¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A prefix that starts with a letter, such as |
|
|
|
An absolute IRI, such as |
Each is a ProvValueError when the Namespace is built, from code or from a reader.
PROV-N refuses the declaration at its token with a ProvNSyntaxError, and the other
readers raise their format’s error with the model’s error as the cause. A PROV-XML read
raises it at the bundleContent element that declares the prefix, or at the first name
under a prefix the document element declares, and raises the model’s refusal of an xmlns
on the document element at <prov:document>. A prefix is a letter, followed by name
characters or ., and does not end in .. 3.2.3 accepted a prefix that is not a
PN_PREFIX and wrote text that did not read back. The Namespace constructor’s refusals
of an empty prefix and of a blank IRI are a ProvValueError now, from code, which
except ValueError still catches. A prefix or an IRI that is not a string is a
ProvValueError too, where 3.2.3 accepted an integer prefix.
Python values¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A value with a PROV form, such as the |
|
|
|
A finite |
The error is a ProvValueError where the value is asserted. An attribute value has a
PROV form when it is a str, int, float, bool, datetime.datetime, Identifier,
QualifiedName, Literal or record, or an instance of a subclass of one of them, or a
value of a type in the table below, which 3.3.0 stores as a Literal of its datatype in
the form shown. 3.2.x stored a value of any other type as given, and no format could
write it faithfully.
Python type |
Datatype |
Written as |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
days, hours, minutes and seconds, such as |
|
|
|
|
|
standard base64 |
A converted value reads back as the Literal, not as the Python value, from
get_attribute() and from every format, except that an xsd:duration read from RDF comes
back in rdflib’s normalised form, so a zero timedelta reads as P0D
(#620,
Conformance, C10). The readers never build these types unless a
json.load option asks for one. With parse_float=decimal.Decimal, the PROV-JSON and
PROV-JSONLD readers read each bare JSON number with a fraction or an exponent as a
Decimal, which is stored as an xsd:decimal Literal.
An Identifier given as a value is stored as a Literal typed xsd:anyURI, the form
every writer wrote for it and every reader now builds, so isinstance(value, Identifier)
no longer matches a stored value. A QualifiedName value is unchanged. 3.2.3 did not
warn about this.
Qualified names¶
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A declared prefix, such as |
|
|
|
The qualified name, |
|
|
|
A qualified name with a declared prefix, or a string if the value is not a name. |
|
|
|
A declared prefix. |
|
|
|
The qualified name, |
|
|
|
A prefixed key, or |
In PROV-N the error is a ProvNSyntaxError at the value or the name, and the lenient
profile skips the statement with a ProvWarning. In PROV-JSON it is a ProvJSONException
that names the record and the attribute. From code it is a
ProvExceptionInvalidQualifiedName raised where the value is asserted, which names the
attribute after the value: Invalid Qualified Name: zz:x, the qualified name 'zz:x' of ex:k does not resolve, because its prefix 'zz' is not declared. The PROV-JSONLD and
PROV-O readers raise ProvJSONLDException and ProvRDFException with it as the cause.
A value typed prov:QUALIFIED_NAME or xsd:QName holds a qualified name, and so does
'zz:x' in PROV-N. prov splits its text at the first : and looks the prefix up in the
declarations in scope, the bundle’s and then the document’s, and never matches a declared
namespace IRI against the text. So http://example.org/x is the prefix http, rejected
unless a prefix named http is declared, and a prefix the document declares is found from
a bundle before the bundle’s own namespaces could compact the text. A value whose prefix
is declared is stored as a QualifiedName, in code and from every format. prov 3.2.x
kept a value whose prefix was not declared as a Literal, and read a full IRI as a name in
a declared namespace the IRI starts with. The last two rows are positions 3.2.3 read
without a warning.
A formal attribute is one of the record’s arguments in PROV-N, such as prov:entity or
prov:activity. The PROV-JSON reader applies the same rule to a record key, a bundle key,
an attribute name, a formal attribute value other than a time and a hadMember member.
3.2.3 dropped a formal attribute value or a hadMember member that did not resolve, with a
warning.
In PROV-JSONLD, the context marks the values of type, role and location as IRIs, so
a bare string there is an IRI, as a {"@id": …} node reference is: one a declared
namespace compacts is the qualified name, and one no namespace compacts is rejected with
ProvJSONLDException, as a node reference is. 3.2.3 kept that string as an xsd:QName
Literal and warned that 3.3.0 would reject it, and read a node reference no namespace
compacts as an Identifier. A reference to a blank node, {"@id": "_:b1"} or
"_:b1", names no record, so the reader drops that value with a ProvWarning. An
xsd:QName or prov:QUALIFIED_NAME value object holding a full IRI or an undeclared
prefix is rejected by the model, and the reader’s error names the record.
In code, a plain string given as an identifier, such as
bundle.entity("http://example.org/e1"), still compacts when no prefix named http is
declared.
The xsi prefix¶
prov 3.2.x accepts these without a warning.
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A declaration, such as |
|
|
|
Another prefix for that namespace. |
An xsi: identifier with no declaration raises ProvElementIdentifierRequired instead
in code, and the PROV-JSONLD reader raises ProvJSONLDException with it as the cause.
PROV-XML documents are unchanged, because the PROV-XML codec reads and writes xsi:type
itself.
prov 3.2.x writes xsi: names to PROV-JSON and PROV-JSONLD without an xsi entry in
the prefix map, so add an xsi entry for http://www.w3.org/2001/XMLSchema-instance to
such a file before reading it with 3.3.0.
Unification¶
prov 3.3.0 raises ProvUnificationError for each of these. The
message starts with the identifier of the records that do not unify.
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The same plan on every record with that identifier, or a plan on none of them. |
|
|
|
The same activity on every record with that identifier, or an activity on none of them. |
|
|
|
The same generation on every record with that identifier, or the activity of the derivation on each. The same holds for |
unified() merges records that share an identifier, and an absent formal attribute
usually unifies with any value. PROV-CONSTRAINTS makes a few positions non-expandable,
where an absent value means “none” and unifies only with another absent value. They are
the plan of an association, the activity of a derivation, and the generation and usage
of a derivation that names no activity. wasRevisionOf, wasQuotedFrom and
hadPrimarySource are derivations and follow the same rule. prov 3.2.3 merged an
absent value in one of these positions with a present one, kept the present value and
warned.
unified() raises for the first such position in the order of the statement’s
arguments. A conflict between two present values, in any position, raises first.
prov_to_graph() flattens the document and then calls unified(), so it raises for the
same documents and for a document whose bundles disagree about a shared identifier.
prov_to_dot() draws such a document as asserted, without merging any records.
Equality¶
== on records, bundles, documents and Literal values follows one rule in 3.3.0, and
3.2.3 did not warn about any of these changes. Code that compares documents, holds records
in a set or asserts the type get_attribute() returns may see a different result.
Compared |
3.2.3 |
3.3.0 |
|---|---|---|
A relation without an identifier and the same relation with one |
equal when the anonymous one is on the left |
never equal (#525) |
A document and a copy with an extra bundle |
equal from the left, unequal from the right |
unequal from either side |
|
two values |
one value (#527) |
|
one value, through |
two values: |
|
two values |
one value |
|
read as |
kept as a |
|
|
|
|
|
|
A PROV-O relation written as a blank node, under a default namespace |
identifier from the blank node’s label |
no identifier |
|
both kept |
Two attribute values are the same value when they have the same datatype and the same
value in that datatype’s value space. xsd:boolean, xsd:decimal, xsd:double,
xsd:float and xsd:integer with its subtypes have a value space here; every other
datatype compares by text, and a text outside its datatype’s lexical space compares as
text. An xsd:float is keyed as a Python float, so two texts that differ only beyond
single precision are two values. A NaN is one value with itself. A bundle compares as a set of records, not by its
identifier; a document matches its bundles by identifier, and a bundle and a document are
never equal, from either side. A plain Python value takes the datatype the writers give
it, so 5 and "5" %% xsd:long stay two values. hash() follows == on records and
literals. prov-compare applies the same ==, so its exit code changes for the same
documents.
get_records(ProvEntity) returns a list where 3.2.3 returned a one-shot iterator
(#491); write next(iter(...)) to take
the first record.
PROV-JSON¶
prov 3.3.0 raises ProvJSONException for each of these. The message starts with the
record identifier and the attribute name, as in record 'ex:g', attribute 'prov:entity': the value is written as an array; a repeated key names only the key, because Python’s
JSON parser reports no position for it.
Error |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
One key. For an attribute with several values, an array, |
|
|
|
A full |
|
|
|
The value, |
|
|
|
No attribute. |
|
|
|
A qualified name, as in |
|
|
|
Only |
|
|
|
A string, as in |
|
|
|
A qualified name, as in |
|
|
|
A declared prefix, such as |
A number in $ typed the way prov 2.x and earlier wrote it still reads. That is an
integer as xsd:int or xsd:long, or a number with a fraction or exponent as
xsd:double, in a value object with no lang, so {"$": 1, "type": "xsd:int"} and
{"$": 1.5, "type": "xsd:double"} read as they did.
The error for a value that is not a qualified name covers a hadMember member and a
formal attribute other than a time. A time written as a value object goes through the
value-object checks first, so an unknown key, a $ or type that is not a string, or
an undeclared datatype gives the message in the table; a value object of any other
datatype, or with a lang, is not an xsd:dateTime.
A record key has no attribute, so its message starts with record 'ex:g' and a colon.
A bundle key starts with bundle 'ex:b' and a colon.