Python 2 happily reads UTF-8 files in text mode, but Python 3 requires
either UTF-8 locale or an explicit encoding passed to open(). Commit
d4e5ec877ca fixed this by setting the en_US.UTF-8 locale. Falls apart
when the locale isn't be available.
Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use
binary mode instead, with manual conversion from bytes to str. Works,
but opening with an explicit encoding is simpler, so do that.
Since Python 2's open() doesn't support the encoding parameter, we
need to suppress it with a version check.
Backports commit de685ae5e9a4b523513033bd6cadc8187a227170 from qemu
It often happens that just a few discriminator values imply extra data in
a flat union. Existing checks did not make possible to leave other values
uncovered. Such cases had to be worked around by either stating a dummy
(empty) type or introducing another (subset) discriminator enumeration.
Both options create redundant entities in qapi files for little profit.
With this patch it is not necessary anymore to add designated union
fields for every possible value of a discriminator enumeration.
Backports commit 800877bb1639d38ffaebe312a37b61c66bb10c83 from qemu
The event generator produces an enum, and put it in the last visited
module. It fits better in the main module, since it's the set of all
visited events, from all modules.
Backports commit f030ffd39d6c1ea8fff281be5e4b19c819d7ce10 from qemu
Now that we can safely call QOBJECT() on QObject * as well as its
subtypes, we can have macros qobject_ref() / qobject_unref() that work
everywhere instead of having to use QINCREF() / QDECREF() for QObject
and qobject_incref() / qobject_decref() for its subtypes.
The replacement is mechanical, except I broke a long line, and added a
cast in monitor_qmp_cleanup_req_queue_locked(). Unlike
qobject_decref(), qobject_unref() doesn't accept void *.
Note that the new macros evaluate their argument exactly once, thus no
need to shout them.
Backports commit cb3e7f08aeaab0ab13e629ce8496dca150a449ba from qemu
Move qapi-schema.json to qapi/, so it's next to its modules, and all
files get generated to qapi/, not just the ones generated for modules.
Consistently name the generated files qapi-MODULE.EXT:
qmp-commands.[ch] become qapi-commands.[ch], qapi-event.[ch] become
qapi-events.[ch], and qmp-introspect.[ch] become qapi-introspect.[ch].
This gets rid of the temporary hacks in scripts/qapi/commands.py,
scripts/qapi/events.py, and scripts/qapi/common.py.
Backports commit eb815e248f50cde9ab86eddd57eca5019b71ca78 from qemu
In my "build everything" tree, a change to the types in
qapi-schema.json triggers a recompile of about 4800 out of 5100
objects.
The previous commit split up qmp-commands.h, qmp-event.h, qmp-visit.h,
qapi-types.h. Each of these headers still includes all its shards.
Reduce compile time by including just the shards we actually need.
To illustrate the benefits: adding a type to qapi/migration.json now
recompiles some 2300 instead of 4800 objects. The next commit will
improve it further.
Backports commit 9af2398977a78d37bf184d6ff6bd04c72bfbf006 from qemu
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth. These monolithic headers get included all
over the place. In my "build everything" tree, adding a QAPI type
recompiles about 4800 out of 5100 objects.
We wouldn't write such monolithic headers by hand. It stands to
reason that we shouldn't generate them, either.
Split up generated qapi-types.h to mirror the schema's modular
structure: one header per module. Name the main module's header
qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h.
Mirror the schema's includes in the headers, so that qapi-types.h gets
you everything exactly as before. If you need less, you can include
one or more of the sub-module headers. To be exploited shortly.
Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h,
qmp-commands.c, qapi-event.h, qapi-event.c the same way.
qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic.
The split of qmp-commands.c duplicates static helper function
qmp_marshal_output_str() in qapi-commands-char.c and
qapi-commands-misc.c. This happens when commands returning the same
type occur in multiple modules. Not worth avoiding.
Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and
qmp-commands.[ch] to qapi-commands.[ch], name the shards that way
already, to reduce churn. This requires temporary hacks in
commands.py and events.py. Similarly, c_name() must temporarily
be taught to munge '/' in common.py. They'll go away with the rename.
Backports commit 252dc3105fc494182e236e97fe20f2d6b1d652cb from qemu
guardname() fails to return a valid C identifier for arguments
containing anything but [A-Za-z0-9_.-']. Fix that. Don't bother
protecting ticklish identifiers; header guards are all-caps, and no
ticklish identifiers are.
Backports commit f9c146399dabefb8cd13c9c467a9e710af15ea70 from qemu
Linking code from multiple separate QAPI schemata into the same
program is possible, but involves some weirdness around built-in
types:
* We generate code for built-in types into .c only with option
--builtins. The user is responsible for generating code for exactly
one QAPI schema per program with --builtins.
* We generate code for built-in types into .h regardless of
--builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN. Because all
copies of this code are exactly the same, including any combination
of these headers works.
Replace this contraption by something more conventional: generate code
for built-in types into their very own files: qapi-builtin-types.c,
qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but
only with --builtins. Obey --output-dir, but ignore --prefix for
them.
Make qapi-types.h include qapi-builtin-types.h. With multiple
schemata you now have multiple qapi-types.[ch], but only one
qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and
qapi-builtin-visit.[ch].
Bonus: if all you need is built-in stuff, you can include a much
smaller header. To be exploited shortly.
Backports commit cdb6610ae4283720037bae2af1f78bd40eb5fe71 from qemu
The use of QAPIGen is rather shallow so far: most of the output
accumulation is not converted. Take the next step: convert output
accumulation in the code-generating visitor classes. Helper functions
outside these classes are not converted.
Backports commit 71b3f0459c460c9e16a47372ccddbfa6e2c7aadf from qemu
The include directive permits modular QAPI schemata, but the generated
code is monolithic all the same. To permit generating modular code,
the front end needs to pass more information on inclusions to the back
ends. The commit before last added the necessary information to the
parse tree. This commit adds it to the intermediate representation
and its QAPISchemaVisitor. A later commit will use this to to
generate modular code.
New entity QAPISchemaInclude represents inclusions. Call new visitor
method visit_include() for it, so visitors can see the sub-modules a
module includes.
Note that unlike other entities, QAPISchemaInclude has no name, and is
therefore not added to entity_dict.
New QAPISchemaEntity attribute @module names the entity's source file.
Call new visitor method visit_module() when it changes during a visit,
so visitors can keep track of the module being visited.
Backports commit cf40a0a5c2e1091846974cc8cc95a60e0b1db4af from qemu
The generators' conversion to visitors (merge commit 9e72681d16)
changed the processing order of entities from source order to
alphabetical order. The next commit needs source order, so change it
back.
Backports commit 8a84767cc4f7e00e5dd62435c32be9e7d2cbe4d3 from qemu
The parse tree is a list of expressions. Except include expressions
currently get replaced by the included file's parse tree.
Instead of throwing away the include expression, keep it with the file
name expanded so you don't have to track the including file's
directory to make sense of it.
A future commit will put this include expression to use.
Backports commit 97f0249474d19c1d60fb9d934c8bc08625a619ca from qemu
Error messages print absolute file names of included files even if the
user gave a relative one on the command line:
$ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
Improve this to
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from tests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json
The error message when an include file can't be opened prints the
include directive's file name, which is relative to the including
file. Change this to print the file name relative to the working
directory. Visible in tests/qapi-schema/include-no-file.err.
Backports commit af97502ce9c648ae5c746b9e562d6e4586f02eee from qemu
A massive number of objects depends on QAPI-generated headers. In my
"build everything" tree, it's roughly 4800 out of 5100. This is
particularly annoying when only some of the generated files change,
say for a doc fix.
Improve qapi-gen.py to touch its output files only if they actually
change. Rebuild time for a QAPI doc fix drops from many minutes to a
few seconds. Rebuilds get faster for certain code changes, too. For
instance, adding a simple QMP event now recompiles less than 200
instead of 4800 objects. But adding a QAPI type is as bad as ever;
we've clearly got more work to do.
Backports commit 907b846653fb3757bf2ab98d6d66f92df34d875f from qemu
Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files. Similar for qga/qapi-schema.json. This is
silly. Replace the six programs by a single program that spits out
all eleven files.
The programs become modules in new Python package qapi, along with the
helper library. This requires moving them to scripts/qapi/. While
moving them, consistently drop executable mode bits.
Backports commit fb0bc835e56b894cbc7236294921e5393c786ad8 from qemu
