visualize is failing #1652

code-surfer · 2022-01-05T16:13:13Z

When I try to run the visualize task, it always fails, the second time differently than the first time:

% rm -r out

% ./millw show visualize __ 
[1/1] show 
[1/1] show > [3/125] visualize 
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Warning: Nashorn engine is planned to be removed from a future JDK release
1 targets failed
show 1 targets failed
visualize java.lang.reflect.InvocationTargetException
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:566)
    mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)
guru.nidi.graphviz.engine.GraphvizException: None of the provided engines could be initialized.
    guru.nidi.graphviz.engine.Graphviz.getEngine(Graphviz.java:162)
    guru.nidi.graphviz.engine.Graphviz.doExecute(Graphviz.java:312)
    guru.nidi.graphviz.engine.Graphviz.execute(Graphviz.java:292)
    guru.nidi.graphviz.engine.Renderer.execute(Renderer.java:139)
    guru.nidi.graphviz.engine.Renderer.toFile(Renderer.java:59)
    mill.main.graphviz.GraphvizTools$.$anonfun$apply$19(GraphvizTools.scala:67)
    scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
    scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
    scala.collection.AbstractIterable.foreach(Iterable.scala:919)
    scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
    mill.main.graphviz.GraphvizTools$.apply(GraphvizTools.scala:66)
    mill.main.graphviz.GraphvizTools.apply(GraphvizTools.scala)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:566)
    mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)

% ./millw show visualize __
[1/1] show 
[1/1] show > [3/125] visualize 
[Fatal Error] :1:1: Premature end of file.
Jan 05, 2022 3:47:04 PM com.kitfox.svg.SVGUniverse loadSVG
WARNING: Error processing svgSalamander://graph/
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1243)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635)
        at com.kitfox.svg.SVGUniverse.loadSVG(SVGUniverse.java:608)
        at com.kitfox.svg.SVGUniverse.loadSVG(SVGUniverse.java:545)
        at com.kitfox.svg.SVGUniverse.loadSVG(SVGUniverse.java:502)
        at guru.nidi.graphviz.engine.SalamanderRasterizer.createDiagram(SalamanderRasterizer.java:43)
        at guru.nidi.graphviz.engine.SalamanderRasterizer.doRasterize(SalamanderRasterizer.java:31)
        at guru.nidi.graphviz.engine.SvgRasterizer.rasterize(SvgRasterizer.java:40)
        at guru.nidi.graphviz.engine.Renderer.toImage(Renderer.java:123)
        at guru.nidi.graphviz.engine.Renderer.toFile(Renderer.java:75)
        at guru.nidi.graphviz.engine.Renderer.lambda$toFile$4(Renderer.java:61)
        at guru.nidi.graphviz.engine.EngineResult.mapIO(EngineResult.java:72)
        at guru.nidi.graphviz.engine.Renderer.toFile(Renderer.java:59)
        at mill.main.graphviz.GraphvizTools$.$anonfun$apply$19(GraphvizTools.scala:67)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
        at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
        at mill.main.graphviz.GraphvizTools$.apply(GraphvizTools.scala:66)
        at mill.main.graphviz.GraphvizTools.apply(GraphvizTools.scala)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)
        at mill.api.Result$.create(Result.scala:18)
        at mill.main.VisualizeModule.$anonfun$worker$5(VisualizeModule.scala:46)
        at java.base/java.lang.Thread.run(Thread.java:829)

1 targets failed
show 1 targets failed
visualize java.lang.reflect.InvocationTargetException
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:566)
    mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)
java.lang.NullPointerException
    guru.nidi.graphviz.engine.SalamanderRasterizer.createDiagram(SalamanderRasterizer.java:45)
    guru.nidi.graphviz.engine.SalamanderRasterizer.doRasterize(SalamanderRasterizer.java:31)
    guru.nidi.graphviz.engine.SvgRasterizer.rasterize(SvgRasterizer.java:40)
    guru.nidi.graphviz.engine.Renderer.toImage(Renderer.java:123)
    guru.nidi.graphviz.engine.Renderer.toFile(Renderer.java:75)
    guru.nidi.graphviz.engine.Renderer.lambda$toFile$4(Renderer.java:61)
    guru.nidi.graphviz.engine.EngineResult.mapIO(EngineResult.java:72)
    guru.nidi.graphviz.engine.Renderer.toFile(Renderer.java:59)
    mill.main.graphviz.GraphvizTools$.$anonfun$apply$19(GraphvizTools.scala:67)
    scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
    scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
    scala.collection.AbstractIterable.foreach(Iterable.scala:919)
    scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
    mill.main.graphviz.GraphvizTools$.apply(GraphvizTools.scala:66)
    mill.main.graphviz.GraphvizTools.apply(GraphvizTools.scala)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:566)
    mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)

This is with both 0.9.9 and 0.9.11 on Linux.

It looks as if this was reported before in #363, but I wonder what the resolution was. Passing the -i option to mill (as seems to be indicated in #363's last comment) does not help.

The text was updated successfully, but these errors were encountered:

code-surfer · 2022-01-19T09:00:54Z

I tried mill 0.10.0 and got the same failures.

lefou · 2022-01-19T09:40:57Z

Please add a the output mill --version for more context. Also, looks like upstream has still issues which look related. nidi3/graphviz-java#181

code-surfer · 2022-01-19T11:19:24Z

% ./millw --version        
Mill Build Tool version 0.10.0
Java version: 11.0.13, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "Linux", version: 5.4.0-94-generic, arch: amd64

sake92 · 2022-05-03T20:44:03Z

There are a few current issues with visualize task/graphviz (if we want to support JDKs 8 and newer):

JGraphT is using "too new" version, 1.5.0+ do not support JDK 8
https://github.com/jgrapht/jgrapht#introduction
Nashorn JS engine is removed in JDK 15
we cannot force users to install graphviz cmdline tool separately!?

All in all, deps should look something like this:

val graphvizJavaMin = ivy"guru.nidi:graphviz-java-min-deps:0.18.1"
val svgSalamander = ivy"guru.nidi.com.kitfox:svgSalamander:1.1.3" // needed for SVGs
val vizJsJava = ivy"org.webjars.npm:viz.js-graphviz-java:2.1.3" // JS impl of graphviz
val graalJS = ivy"org.graalvm.js:js:21.0.0" // js engine for jvm
val jgraphtCore = ivy"org.jgrapht:jgrapht-core:1.4.0" // 1.5.0+ dont support JDK 8

I tried to do the upgrade, but getting a StackOverflowError:

1 targets failed
show 1 targets failed
visualize java.lang.reflect.InvocationTargetException
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:568)
mill.main.VisualizeModule.$anonfun$worker$6(VisualizeModule.scala:50)
java.lang.StackOverflowError
guru.nidi.graphviz.engine.Graphviz.useEngine(Graphviz.java:123)
guru.nidi.graphviz.engine.Graphviz.useDefaultEngines(Graphviz.java:112)
guru.nidi.graphviz.engine.Graphviz.useEngine(Graphviz.java:124)

It is almost like graalJS isn't included on classpath or something.
I did add all those to graphviz.ivyDeps.. and made an mill installLocal.

Btw I am not sure how current impl works, since I cant find guru.nidi.graphviz.engine.Graphviz inside the Mill JAR.

lefou · 2022-05-03T20:51:06Z

From looking at all these deps, we should use a dedicated worker with isolated classloader, to avoid loading all this stuff into Mill core.

I think it would probably be also ok to just use a locally installed graphviz binary to render the graph, if the Java runtime isn't any longer able to run it.

lefou · 2022-05-03T20:52:55Z

I mean, the graph is nice, but we could also just spit out the dot file, couldn't we?

sake92 · 2022-05-03T21:04:19Z

Yeah, that would be the best probably:

dot file by default
svg, jpg etc via plugin/module

sake92 · 2022-05-12T16:28:05Z

Seems like I was overcomplicating things a bit...
The solution is simpler than I thought, they have a bundled dependency that works!

Fixes #1652 1. use graphviz-java-all-j2v8 instead of graphviz-java (now as transitive dependency) Following instructions from graphviz-java [docs](https://github.com/nidi3/graphviz-java#maven ) they say: > gradle does not support this way of defining a dependency > Instead of graphviz-java there are two alternative dependencies that can be used: > - graphviz-java-all-j2v8 additionally contains dependencies to all J2V8 platforms. So the same application can run on Linux, Mac OS X and Windows. > - graphviz-java-min-deps contains only dependencies that are absolutely necessary. All other dependencies are marked as optional and must added manually. See the [pom.xml](https://github.com/nidi3/graphviz-java/blob/master/graphviz-java-min-deps/pom.xml#L64-L90) for details. 2. downgrade jgrapht since 1.5.0+ dont support JDK8 https://github.com/jgrapht/jgrapht#dependencies This adds around 3MB of extra dependencies. The upside is that it works on all JDK: 8, 11 and 17! (tested locally on win10) Pull request: #1872

sake92 mentioned this issue May 12, 2022

Fix visualize command #1872

Merged

lefou closed this as completed in #1872 May 12, 2022

lefou added this to the after 0.10.4 milestone May 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

visualize is failing #1652

visualize is failing #1652

code-surfer commented Jan 5, 2022

code-surfer commented Jan 19, 2022

lefou commented Jan 19, 2022

code-surfer commented Jan 19, 2022

sake92 commented May 3, 2022

lefou commented May 3, 2022

lefou commented May 3, 2022

sake92 commented May 3, 2022

sake92 commented May 12, 2022

visualize is failing #1652

visualize is failing #1652

Comments

code-surfer commented Jan 5, 2022

code-surfer commented Jan 19, 2022

lefou commented Jan 19, 2022

code-surfer commented Jan 19, 2022

sake92 commented May 3, 2022

lefou commented May 3, 2022

lefou commented May 3, 2022

sake92 commented May 3, 2022

sake92 commented May 12, 2022