Prepare for prov 3.3.0¶
prov 3.3.0 rejects some input that 3.2.x reads, and reads a few other inputs
differently. From 3.2.3, prov warns about each such input with a
ProvFutureRejectionWarning where it reads the input, and
still reads it as before. This guide shows how to find every such input in your
documents and code.
Turn the warnings into errors¶
ProvFutureRejectionWarning is a subclass of ProvWarning.
An error filter on the subclass raises at the first such message and leaves the
other ProvWarnings alone, such as the statements the lenient PROV-N profile
skips.
When running pytest:
pytest -W error::prov.model.ProvFutureRejectionWarning
In a pytest configuration:
[tool.pytest.ini_options]
filterwarnings = ["error::prov.model.ProvFutureRejectionWarning"]
In code:
import warnings
from prov.model import ProvFutureRejectionWarning
warnings.simplefilter("error", ProvFutureRejectionWarning)
Python’s own -W option and the PYTHONWARNINGS environment variable cannot
name this class, because Python reads them before it can import prov. Use
pytest’s option, a pytest configuration or a filter in code instead.
prov 3.3.0 keeps the class, so these filters keep working after the upgrade.
Read the message¶
Each message ends with what 3.3.0 does instead and the number of the GitHub issue
that tracks the rule, such as “prov 3.3.0 will reject this (#529)”. A message from a
reader starts with the position of the input: the line and column in PROV-N, or the
record identifier and attribute name in PROV-JSON. A message about a language tag, a
local part, a Python value or a time offset beyond ±14:00 comes from the model,
wherever the value was read or built, so it names the value instead. So does a
message about a qualified-name value built in code or read from PROV-JSONLD or RDF.
A message from unified() starts with the identifier of the records it merges.
A PROV-N read emits its messages in three groups: the model’s messages, as the
parse runs, then the reader’s messages about whitespace, then the reader’s other
messages. The reader’s messages wait until the parse ends, so a read that fails
with ProvNSyntaxError emits none of them. The messages therefore do not follow
the order of the input, and under an error filter the first one raised may not
be the first problem in the file.
A filter on the issue number silences every case that issue tracks and leaves
every other case an error. Python tries the filter added last first, so add the
ignore filter after the error filter. A message that names a value can contain
a line break, so start the pattern with (?s), which makes . match a line break
too. This example ignores the cases that issue #529 tracks.
import warnings
from prov.model import ProvFutureRejectionWarning
warnings.simplefilter("error", ProvFutureRejectionWarning)
warnings.filterwarnings(
"ignore", message=r"(?s).*\(#529\)$", category=ProvFutureRejectionWarning
)
PROV-N declarations¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
One |
|
|
|
A prefix that starts with a letter, such as |
|
|
|
Add |
|
|
|
The |
|
|
|
No declaration. Both prefixes are predeclared. |
prov 3.2.2 and earlier wrote xsi: names without a prefix xsi line. Reading
such a file and writing it again with prov 3.2.3 adds the declaration.
PROV-N whitespace¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A space, tab, CR or LF. |
|
|
|
A space, tab, CR or LF. 3.3.0 reads U+1680 as part of a name. |
The same warning names any other Unicode whitespace character between tokens, such as U+000C (form feed) or U+2028 (line separator). Whitespace inside a string, an IRI or a comment does not warn.
PROV-N statements¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The value as the argument, |
|
|
|
Every statement before the first bundle. |
|
|
|
|
|
|
|
|
The identifier and attribute-block warnings name whichever of alternateOf,
specializationOf and hadMember carries one. The default and lenient
profiles accept the three strict-profile constructs in this table without a
warning.
Times¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The same instant with an offset from -14:00 to +14:00, such as |
|
|
|
The same instant with an offset from -14:00 to +14:00, such as |
The first warning covers every time of a statement: the time of a relation, an
activity’s start and end times, and prov:time given as an attribute, from any
format or from code. An xsd:dateTime value of any other attribute is not a
time of the statement, so 3.3.0 does not reject it. 3.3.0 reads the value as a
Literal with its text, not as a datetime.
Each warning fires when the time is first asserted. unified(), flattened(),
update(), add_record() and copies of the record or the document do not warn
again.
Local parts¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A local part without the character, such as |
A local part cannot hold a space, a control character or another character
that PROV-N can write only by percent-encoding it, such as " or |. Nor can
it hold a % that two hexadecimal digits do not follow, or start with U+00B7
(middle dot), U+203F (undertie), U+2040 (character tie) or a combining mark
from U+0300 to U+036F. The PROV-N writer percent-encodes such a character, so
a PROV-N reader recovers a different IRI. The characters = ' ( ) , : ; [ ]
are written with a backslash instead, and do not warn.
The warning fires when the name is first built. unified(), flattened() and
copies of the document reuse the name and do not warn again. The XML reader
builds each occurrence of a name afresh, so it warns once for each.
Python values¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The |
|
|
|
A value with a PROV form, such as the |
An attribute value has a PROV form when it is a str, int, float, bool,
datetime.datetime, Identifier, QualifiedName, Literal or record, or an
instance of a subclass of one of them. prov 3.2.x stores a value of any other
type as given, and no format can write it faithfully. 3.3.0 stores a value of a
type in the table below as a Literal of its datatype, and rejects a value of
any other type.
Python type |
3.3.0 datatype |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
In 3.3.0, a converted value reads back as the Literal, not as the Python
value. The readers never build these types unless a json.load option asks for
one. With parse_float=decimal.Decimal, for example, the PROV-JSON and
PROV-JSONLD readers read each bare JSON number with a fraction or an exponent
as a Decimal, and warn for each such number.
The warning fires when the value is first asserted. unified(), flattened(),
update(), add_record() and copies of the record or the document do not warn
again.
Qualified names¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
A declared prefix, such as |
|
|
|
The qualified name, |
|
|
|
A qualified name with a declared prefix, or a string if the value is not a name. |
|
|
|
A declared prefix. |
A value typed prov:QUALIFIED_NAME or xsd:QName holds a qualified name, and
so does 'zz:x' in PROV-N. 3.3.0 splits its text at the first : and looks the
prefix up in the declarations in scope. prov 3.2.x keeps a value whose prefix
is not declared as a Literal. It also reads a full IRI typed
prov:QUALIFIED_NAME, or written '…' in PROV-N, as a name in a declared
namespace that the IRI starts with, so http://example.org/x reads as ex:x.
3.3.0 reads that text as the prefix http and rejects it, unless a prefix named
http is declared.
A formal attribute is one of the record’s arguments in PROV-N, such as
prov:entity or prov:activity. The PROV-JSON reader applies the same rule to a
record key, a formal attribute value other than a time, a hadMember member and a
bundle key that is a full IRI. It also warns when it drops a formal attribute
value or a hadMember member that does not resolve, such as
"prov:activity": "zz:a".
The PROV-JSONLD reader warns about an xsd:QName or prov:QUALIFIED_NAME value
object that holds a full IRI. The message uses the model’s wording and has no
position. The PROV-JSONLD context marks the values of type, role and
location as IRIs, so a bare full IRI there that starts with a declared namespace
IRI does not warn. One that starts with no declared namespace IRI is kept as a
Literal and warns under #516, as does a name with an undeclared prefix. A
{"@id": …} node reference stays an IRI and does not warn.
The PROV-N and PROV-JSON readers warn with the position of the value, and the
model does not warn again. The model warns about a value built in code or read
from another format when the value is first asserted, and names the attribute
after the value, as in the qualified name 'zz:x' of ex:k does not resolve.
In code, a plain string given as an identifier, such as
bundle.entity("http://example.org/e1"), still compacts in 3.3.0, and does not
warn.
Unification¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
The same plan on every record with that identifier, or a plan on none of them. |
|
|
|
The same activity on every record with that identifier, or an activity on none of them. |
|
|
|
The same generation on every record with that identifier, or the activity of the derivation on each. The same holds for |
unified() merges records that share an identifier, and an absent formal
attribute usually unifies with any value. PROV-CONSTRAINTS makes a few positions
non-expandable, where an absent value means “none” and unifies only with another
absent value. They are the plan of an association, the activity of a
derivation, and the generation and usage of a derivation that names no
activity. wasRevisionOf, wasQuotedFrom and hadPrimarySource are
derivations and follow the same rule. In 3.2.x unified() still merges an
absent value in one of these positions with a present one and keeps the present
value. 3.3.0 raises ProvUnificationError.
unified() warns once for each set of records of one type that share an
identifier, at the first such position in the order of the statement’s
arguments, which is where 3.3.0 raises.
prov_to_graph() and prov_to_dot() call unified(), so they warn about the
same documents. In 3.3.0 prov_to_graph() raises, and prov_to_dot() draws the
document as asserted, without merging any records.
PROV-JSON¶
Warning |
Issue |
Example |
Write instead |
|---|---|---|---|
|
|
One key. For an attribute with several values, an array, |
|
|
|
A full |
|
|
|
The value, |
|
|
|
No attribute. A null |
|
|
|
A qualified name, as in |
|
|
|
Only |
|
|
|
A string, as in |
|
|
|
A declared prefix, such as |
prov 3.2.x reads a $ that is not a string as its string form, so {"$": null}
reads as the string "None".
When a datatype does not resolve, prov 3.2.x reads the value as a string, or as
a language-tagged string when it has a lang.
A number in $ typed the way prov 2.x and earlier wrote it does not warn. That
is an integer as xsd:int or xsd:long, or a number with a fraction or exponent as
xsd:double, in a value object with no lang.
{"$": 1, "type": "xsd:int"} and {"$": 1.5, "type": "xsd:double"} read in 3.3.0 as
they do now.
The warning for a value that is not a qualified name covers a hadMember member
and a formal attribute other than a time. A time that is neither a string nor an
xsd:dateTime value object already raises ProvJSONException.
The repeated-key warning names only the key, because Python’s JSON parser reports no position for it.
Every other message starts with the record identifier and the attribute name,
as in record 'ex:g', attribute 'prov:entity': the value is written as an array.
A record key has no attribute, so its message starts with record 'ex:g' and a
colon. A bundle key starts with bundle 'ex:b' and a colon. The members of
hadMember are an array by design, so the array itself does not warn.