The Audio Video Annotations Analysis Toolkit
Corpora analysis is a complex task, requiring to learn editors for different file formats and multiple tools, often command-line based, or with programming knowledge prerequisite.
$ATK makes it easy to create pipelines connecting ecosystems to process raw data (automated transcriptions, formats conversion..), and query large corpora of annotations coming from various sources to extract advanced statistics and generate beautiful, always up-to-date charts and timelines.
$ATK is also a flexible converter ; it takes as input XML files describing the style and operations to generate an HTML document, and takes care of exporting only relevant portions of videos and their thumbnail snapshots, minimizing final document size and potential load times if hosted online.
$ATK understands the following file formats
$ATK can also process the following media types
Simply extract the latest release zip
To start $ATK, simply double-click the launcher file avaa-toolkit.exe
The executable is not signed, so Windows will ask for confirmation before starting it.
In case $ATK fails to start, check the troubleshooting section.
On those systems $ATK can be started by running the shell script avaa-toolkit.sh but it must be set as executable first:
Note that at least Java 11 is required, you can install latest version using trusted adoptium.net binaries.
When $ATK is already installed, follow these steps to update:
An editor for $ATK's XML documents is available in the browser.
To begin, start $ATK by running the launcher (avaa-toolkit.exe on Windows or avaa-toolkit.sh on Linux/macOS),
then navigate with your browser to avaa-toolkit.org
If internet is not available, use the provided offline editor in your installation folder (open index.html)
By default, the editor is allowed to create and edit files in the projects folder.
It is possible to add other folders to the editor, just edit avaa-config.xml file.
$ATK is all about querying and filtering annotations. Complex queries can be expressed to extract only specific annotations.
This is done via the SELECT tag, various attributes can be combined to make a curated selection of annotations:
Attributes of type regexp (*-match) have additional options:
When multiple attributes are used, the selection will consist only of the annotations fulfilling all the constraints.
$ATK features an advanced pipeline system easing automation of complex tasks.
A pipeline is created for each section of the document, and initially contains a virtual copy of the corpus and its associated media files.
The corpus and its media files are then modified sequentially by each processor inside the pipeline.
The pipeline can be fed different initial media files, by defining the processor-pipeline-input setting.
The corpus mode is useful to process corpus files directly (audio-anonymization, formats conversion...), while for instance all-assets mode could be used to apply effects only on the exported media of the document intended for sharing with peers.
Processors inside a pipeline (that is for now, a section of the document) are executed one after another, each processor using the results of the previous one to work on.
Complex chains of processors can be built to automate heavy tasks alleviating the burden of manually running each step and verifying its consistency.
Views placed after a processor (in the same section) will inherit its modified media files when exporting clips and snapshots.
This can be helpful extracting annotations from cuts of raw media files, to avoid processing long corpus media file when testing samples ; or preprocessing a media file before it is exported into clips during later views generation.
Processors generating annotations will make these annotations immediately available in the main corpus (and not only for the current pipeline), hence for all subsequent views and processors in the document.
Settings can be modified at any time via the Local Settings block.
It is possible to change the style via CSS. The HTML code generated makes it easy to target specific elements or apply styling rules for the whole page. Each view has its own structure of elements, and a simple "Inspect Element" from browser will reveal selectors.
Styles can be defined directly in the XML file, by using a STYLE tag.
These styles will only apply to this specific HTML document.
<STYLE>
.view-timeline td {
border-color:red;
}
.view-timeline tr.tier-header {
text-align:right;
}
</STYLE>
Styles can be defined in a separate CSS file, that must be placed in the include folder.
All the generated HTML documents will load this file and have these styles in common.
h2 {
color:green;
}
section {
border-left: 2px solid gray;
}
Views generate simple HTML code and try to follow common guidelines so that applying styles is straightforward
Annotations' text labels always have the annotation class, so for instance to change the color of all annotations:
.view .annotation {
color:red;
}
$ATK can also generate PDF, though interactive features like videos or dynamic charts won't work in this format, for obvious reasons.
Chrome (or Chromium) must be installed on the system (alternatively on Windows $ATK will try to use Edge).
Chrome/Edge executable should be detected automatically, if that fails it is required to provide its path in avaa-config.xml
If Chrome/Edge is not available, it is recommended to install Chrome Headless Shell and then provide its path in avaa-config.xml
$ATK is made for the command line and can integrate seamlessly in any tool chain.
##cli##
Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.UnsupportedClassVersionError: org/avaatoolkit/Main has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to X
Solution: Your version of java runtime is outdated, follow these steps
java.net.BindException: Couldn't bind to any port in the range `42042:42042`. at org.glassfish.grizzly.AbstractBindingHandler.bind(AbstractBindingHandler.java) at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java) at org.glassfish.grizzly.http.server.NetworkListener.start(NetworkListener.java:) at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java) at org.avaatoolkit.server.Daemon.start(Daemon.java) at org.avaatoolkit.Main.main(Main.java)
Solution: The toolkit is already started with the --server argument, close it before running a new instance.
Solution: Your firewall has a strict policy regarding localhost port bindings, add a rule to allow localhost:42042
On some operating systems, the installed java runtime might not be up-to-date and prevent $ATK from executing properly.
To run $ATK, at least java 11 is required. To install a valid runtime:
Alternatively using the OpenJDK archives:
Some processors require a full FFmpeg version to work.
When generating really short clips (under 1 sec), it is possible that the clips will consist of only one frozen image.
This is because by default FFmpeg will be instructed to do a copy of the video stream (vcopy), which saves considerable processing time, at the expense of less accurate clipping.
When perfect accuracy is required for clips, it is recommended to force FFmpeg re-encoding, for instance by defining the Setting video-codec = h264
This documentation includes attributions to licensed material such as libraries and software modules.
These notices are written explicitly in each relevant component and for convenience listed again below.
Some modules are not included in $ATK but rather installed on demand whenever a component requires it.
Other modules are included or integrated in $ATK to provide a better overall user experience.
This behavior is indicated by a little icon preceding the license, as well as a tooltip describing its inclusion method.
Additional libraries are packaged with the produced HTML document, and therefore redistributed by the end user.
jQuery simplifies DOM manipulation, some components use it to initialize content in the browser.
D3 has unparalleled flexibility in building custom and dynamic visualizations.
Charts generated by $ATK are actually rendered right in the browser with D3.
GSAP is incredible and we deemed its inclusion valuable for providing a robust interactivity and animation framework for future $ATK components.
Tipped features easy to use and customizable tooltips.
$ATK views sometimes use these tooltips for instance to show snapshots or videos in a small popup when an annotation is clicked or hovered.
FileSaver.js provides a simple interface to save (as a "download") files created directly in the browser.
We believe $ATK components can benefit from the presence of the FileSaver library.
$ATK itself is built with Java, and makes use of various libraries (via Maven) which are compiled into the final JAR executable distributed to the toolkit users.
A library for extracting things from streaming sites, $ATK includes this library to provide an easy API for downloading PeerTube videos.
A FFmpeg CLI Wrapper for Java, used to execute FFmpeg and read progress feedback.
Rhino is the JavaScript engine used to execute all components' scripts.
The best library for parsing command-line arguments.
This library is used to spawn server sockets, and brings WebSocket sessions (that's how the Editor can interact with $ATK).
Jsoup simplifies HTML/XML parsing via a CSS selectors syntax.
An artifact of fully-specified annotations to power static-analysis checks, beginning with nullness analysis.
Jchardet is a Java port of the source from mozilla's automatic charset detection algorithm.
A JNA-based (native) Operating System and Hardware Information library, to get processes details and CPU usage.
Apache Commons is a set of commonly needed features implemented as reusable Java components.
A simple facade abstraction for various logging frameworks.
A reliable, generic, fast and flexible logging framework.
Automate Java boilerplate code via annotations.
OkHttp is an efficient HTTP client.
The MongoDB Synchronous Driver provides an easy API for interacting with a MongoDB Server.
Drew Noakes' excellent library for extracting metadata (Exif/IPTC/XMP...) from image files.
We are currently working on that.