Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: com-lihaoyi/mill
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 0.10.7
Choose a base ref
...
head repository: com-lihaoyi/mill
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 0.10.8
Choose a head ref

Commits on Aug 24, 2022

  1. Update mill-main to 0.10.7 (#2004)

    Pull request: #2004
    scala-steward authored Aug 24, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    9825513 View commit details
  2. docs: add mill-scip to external plugins (#2000)

    Co-authored-by: Tobias Roeser <le.petit.fou@web.de>
    
    Pull request: #2000
    ckipp01 authored Aug 24, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    3b02fb3 View commit details

Commits on Aug 25, 2022

  1. BSP: Forward --debug option to BSP server (#2007)

    This means, we can use the effective value of `Logger.debugEnabled` in BSP.
    
    We now only send debug messages to the BSP client, when debug is enabled.
    
    Pull request: #2007
    lefou authored Aug 25, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    7e9edf4 View commit details

Commits on Aug 29, 2022

  1. fix: don't log out the effective scalacOptions (#2006)

    This pr makes the following change. Debug messages shouldn't go through
    `build/showMessage`. Taken from the BSP [docs](https://build-server-protocol.github.io/docs/specification.html#show-message):
    
    > The show message notification is sent from a server to a client to ask
    > the client to display a particular message in the user interface.
    
    From my experience these are reserved for helpful messages that the user
    should know about. So I changed this to `build/logMessage` which seems
    more appropriate:
    
    > The log message notification is sent from the server to the client to
    > ask the client to log a particular message.
    
    Fixes #2005
    
    Pull request: #2006
    ckipp01 authored Aug 29, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    d944b3c View commit details

Commits on Aug 30, 2022

  1. Support for Scoverage 2.x (#2010)

    This is the first iteration to support Scoverage 2.x in projects which already work with Scoverage 1.x
    
    It does not specifically handle Scala 3.x, which already comes with a scoverage plugin.
    
    Pull request: #2010
    lefou authored Aug 30, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    08ae68d View commit details

Commits on Aug 31, 2022

  1. Update scalajs-env-nodejs to 1.4.0 (#2011)

    Pull request: #2011
    scala-steward authored Aug 31, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    36f5e64 View commit details

Commits on Sep 1, 2022

  1. Copy the full SHA
    7e3ac93 View commit details

Commits on Sep 3, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    1be805a View commit details
  2. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    59d3803 View commit details
  3. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    d6258b5 View commit details

Commits on Sep 5, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    14b01fd View commit details
  2. Fix typo in scaladoc docs (#2023)

    Pull request: #2023
    lolgab authored Sep 5, 2022
    Copy the full SHA
    f1cc485 View commit details

Commits on Sep 6, 2022

  1. Fix issue with lazy val on Scala Native + Scala 3.2 (#2024)

    Scala Native depends on the scala3-library `3.1.3`.
    To work on Scala 3.2 you need to explicitly depend on scala3-library
    for `3.2.0`.
    In Scala.js it is a single artifact generated in the dotty repository,
    so it's platformed. In Scala Native, however, we need to depend on the
    jvm artifact, which is used by the compiler, so we patch
    `scalaLibraryIvyDeps` to be `platformed = false`
    
    Pull request: #2024
    lolgab authored Sep 6, 2022
    Copy the full SHA
    7b9246c View commit details

Commits on Sep 14, 2022

  1. Contrib: Gitlab plugin (#2008)

    This contrib plugin provides support for Gitlab maven package registry publishing / dependencies and gitlab CI/CD. Gitlab does not support http basic authentication, instead a correct http header / token type must be inserted.
    
    Code is inspired code of artifactory and bintray plugins but does not take credentials from command line. Instead it searches token from environment according to configuration (with hopefully sane defaults). By default CI_JOB_TOKEN environment variable is read which provides no-configuration support for gitlab CI/CD.
    
    This works for me (self hosted gitlab 15.2.2-ee). I'm pretty sure though there is some use case that falls through the cracks. I added quite powerful customization mechanism that allows coding over possible limitations of this implementation. Documentation explains usage quite well and code is not too big to digest.
    
    Pull request: #2008
    
    Co-authored-by: Tobias Roeser <le.petit.fou@web.de>
    aheiska and lefou authored Sep 14, 2022
    Copy the full SHA
    1803cb5 View commit details

Commits on Sep 15, 2022

  1. Fixed inconsistency in args parser (#2030)

    Since we support separated arguments, we first separate all arguemnts and
    later valdate and extract each separated group individualy.
    When we get no args at all, we create no separated groups, hence we dont
    have anything to validate and we miss to create a parse error for it.
    
    This fix instead creates an empty first separated group, so we properly
    validate it and detect the case of missing args.
    
    Pull request: #2030
    lefou authored Sep 15, 2022
    Copy the full SHA
    d5a14a8 View commit details

Commits on Sep 18, 2022

  1. Fixed wrong resolver error message for cross modules (#2038)

    Added a test case for nested cross module resolve error.
    
    * Reported via #2027
    
    Pull request: #2038
    lefou authored Sep 18, 2022

    Verified

    This commit was signed with the committer’s verified signature. The key has expired.
    lefou Tobias Roeser
    Copy the full SHA
    5b5e2ac View commit details

Commits on Sep 19, 2022

  1. Scoverage module now checks for invalid version setups

    Added more tests.
    lefou committed Sep 19, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5df924b View commit details
  2. [Docs] Fix import on scalanative example module (#2041)

    Fixes #2040
    
    Pull request: #2041
    carlosedp authored Sep 19, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    c60bc5c View commit details

Commits on Sep 22, 2022

  1. Fix default commands in nested cross modules (#2039)

    This fixes lookup of default commands in (nested) cross modules.
    
    Before this change, default commands in cross modules (defined via
    `TaskModule`) were ignored, which is an bug IMO.
    
    This fixes issue #2027
    
    I took the opportunity to split up some of the gnarly `Resolve` code
    into multiple files. It's still hard to understand, yet a bit more
    navigable.
    
    Pull request: #2039
    lefou authored Sep 22, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    1cb86db View commit details
  2. Update Scala 3.2.0-RC1 to 3.2.0

    lefou committed Sep 22, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    8226ad3 View commit details

Commits on Sep 23, 2022

  1. Import mainargs @main annotation in scripts (#2044)

    This fixed compile issues, when compiled outside of Ammonite.
    
    Pull request: #2044
    lefou authored Sep 23, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    e237120 View commit details

Commits on Sep 27, 2022

  1. Disabled htmlReport test is specific test env

    I don't know why this fails, but htmlReport seems to work in normal usage.
    lefou committed Sep 27, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    ad13acb View commit details

Commits on Sep 28, 2022

  1. Support Scoverage for Scala 3 (#2016)

    This is a follow-up to the work done in 
    #2010 
    to also add support for Scala 3.
    
    Pull request: #2016
    lefou authored Sep 28, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    c9c409e View commit details

Commits on Sep 29, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    203f2e8 View commit details
  2. Update mill-vcs-version to 0.3.0 (#2051)

    Pull request: #2051
    lefou authored Sep 29, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    03a28db View commit details

Commits on Sep 30, 2022

  1. Update Ammonite to 2.5.4-33-0af04a5b (#2052)

    This version supports Scala 2.13.9
    
    Pull request: #2052
    lefou authored Sep 30, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    1dc2cf9 View commit details
  2. Update Scoverage to 2.0.5 (#2053)

    Support for Scala 2.13.9
    
    Teste for Scala 3.2.0 skip `htmlReport` and `xmlReport`, as there are
    issues with missing source roots, probably because of `inline`-ing.
    
    Pull request: #2053
    lefou authored Sep 30, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    625ad7f View commit details
  3. Add more release versions to Mima checks (#2054)

    Pull request: #2054
    lefou authored Sep 30, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    45e3871 View commit details

Commits on Oct 4, 2022

  1. Update zinc to 1.7.2 (#2056)

    Pull request: #2056
    lefou authored Oct 4, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    0f87161 View commit details

Commits on Oct 6, 2022

  1. Retry coursier artifact loading when "checksum not found" (#2058)

    We reuse the same retry logic as we do for retrying concurrent download
    issues.
    
    Fix #1910
    
    Pull request: #2058
    lefou authored Oct 6, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    f96162e View commit details
  2. Update junitsocket to 2.5.2 (#2057)

    Pull request: #2057
    lefou authored Oct 6, 2022
    Copy the full SHA
    8751bd1 View commit details

Commits on Oct 8, 2022

  1. Show gpg output in server/client in publish (fix #387) (#2059)

    Avoid using `os.Inherit` which is known to break the output in
    server/client mode.
    
    Fixes #387
    
    Checked manually and the errors now appear with and without `-i`.
    
    Pull request: #2059
    lolgab authored Oct 8, 2022
    Copy the full SHA
    c985ef8 View commit details
  2. Bump zinc and account for diagnostic code (#1912)

    Newer zinc brings in is the changes to `Problem` I made in
    sbt/sbt#6874 that expose the diagnostic code of
    the diagnostic coming from dotc. I have been doing some work on that on
    the compiler side in scala/scala3#15565 and
    wanted to try it out with Mill.
    
    I tried to mimic the way you currently have it set up, so let me know if
    it's not the direction you'd want to go. However, the idea here would be
    that the diagnostic code is forwarded when diagnostics are published via
    BSP so that Metals could then capture that code and know what code
    actions to offer. You can see more of the big picture in
    scala/scala3#14904.
    
    Pull request: #1912
    ckipp01 authored Oct 8, 2022
    Copy the full SHA
    0ec0e9e View commit details

Commits on Oct 10, 2022

  1. Update semanticdb and trees to 4.6.0 (#2061)

    Pull request: #2061
    lefou authored Oct 10, 2022
    Copy the full SHA
    be9b330 View commit details
  2. Prepared release 0.10.8

    lefou committed Oct 10, 2022
    Copy the full SHA
    26bf9c5 View commit details
Showing with 1,815 additions and 433 deletions.
  1. +1 −1 .mill-version
  2. +8 −4 bsp/src/mill/bsp/BSP.scala
  3. +1 −1 bsp/src/mill/bsp/BspCompileProblemReporter.scala
  4. +4 −2 bsp/src/mill/bsp/MillBspLogger.scala
  5. +22 −0 bsp/test/src/BspInstallDebugTests.scala
  6. +4 −3 bsp/test/src/BspInstallTests.scala
  7. +67 −27 build.sc
  8. +1 −0 ci/shared.sc
  9. +1 −0 ci/upload.sc
  10. +11 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabAuthHeaders.scala
  11. +32 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabHttpApi.scala
  12. +34 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabPackageRepository.scala
  13. +107 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabPublishModule.scala
  14. +65 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabPublisher.scala
  15. +106 −0 contrib/gitlab/src/mill/contrib/gitlab/GitlabTokenLookup.scala
  16. +109 −0 contrib/gitlab/test/src/mill/contrib/gitlab/GitlabTests.scala
  17. +24 −2 contrib/scoverage/api/src/ScoverageReportWorkerApi.scala
  18. +127 −15 contrib/scoverage/src/ScoverageModule.scala
  19. +1 −1 contrib/scoverage/src/ScoverageReport.scala
  20. +3 −2 contrib/scoverage/test/resources/hello-world/core/test/src/GreetSpec.scala
  21. +193 −46 contrib/scoverage/test/src/HelloWorldTests.scala
  22. +7 −2 contrib/scoverage/worker/src/ScoverageReportWorkerImpl.scala
  23. +46 −0 contrib/scoverage/worker2/src/ScoverageReportWorkerImpl.scala
  24. +5 −5 docs/antora/antora.yml
  25. +1 −0 docs/antora/modules/ROOT/nav.adoc
  26. +1 −1 docs/antora/modules/ROOT/pages/Common_Project_Layouts.adoc
  27. +1 −1 docs/antora/modules/ROOT/pages/Configuring_Mill.adoc
  28. +1 −0 docs/antora/modules/ROOT/pages/Contrib_Plugins.adoc
  29. +220 −0 docs/antora/modules/ROOT/pages/Plugin_Gitlab.adoc
  30. +72 −0 docs/antora/modules/ROOT/pages/Thirdparty_Plugins.adoc
  31. BIN index.scip
  32. +12 −0 main/api/src/mill/api/CompileProblemReporter.scala
  33. +1 −1 main/api/src/mill/api/Logger.scala
  34. +1 −1 main/core/src/mill/define/ParseArgs.scala
  35. +1 −0 main/core/src/mill/eval/Evaluator.scala
  36. +28 −0 main/src/mill/main/LevenshteinDistance.scala
  37. +48 −270 main/src/mill/main/Resolve.scala
  38. +80 −0 main/src/mill/main/ResolveMetadata.scala
  39. +75 −0 main/src/mill/main/ResolveSegments.scala
  40. +82 −0 main/src/mill/main/ResolveTasks.scala
  41. +14 −7 main/src/mill/modules/Jvm.scala
  42. +14 −2 main/test/src/eval/CrossTests.scala
  43. +46 −5 main/test/src/main/MainTests.scala
  44. +1 −0 main/test/src/util/ParseArgsTest.scala
  45. +21 −1 main/test/src/util/TestGraphs.scala
  46. +26 −2 readme.adoc
  47. +4 −0 scalalib/src/PublishModule.scala
  48. +31 −3 scalalib/src/publish/SonatypePublisher.scala
  49. +24 −24 scalalib/test/src/HelloWorldTests.scala
  50. +12 −0 scalalib/worker/src/mill/scalalib/worker/ZincDiagnosticCode.scala
  51. +5 −0 scalalib/worker/src/mill/scalalib/worker/ZincProblem.scala
  52. +1 −0 scalalib/worker/src/mill/scalalib/worker/ZincWorkerImpl.scala
  53. +8 −2 scalanativelib/src/ScalaNativeModule.scala
  54. +3 −0 scalanativelib/test/resources/hello-native-world/src/Main.scala
  55. +2 −2 scalanativelib/test/src/HelloNativeWorldTests.scala
2 changes: 1 addition & 1 deletion .mill-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.10.5
0.10.7
12 changes: 8 additions & 4 deletions bsp/src/mill/bsp/BSP.scala
Original file line number Diff line number Diff line change
@@ -48,11 +48,16 @@ object BSP extends ExternalModule {
val bspFile = bspDirectory / s"${serverName}.json"
if (os.exists(bspFile)) T.log.info(s"Overwriting BSP connection file: ${bspFile}")
else T.log.info(s"Creating BSP connection file: ${bspFile}")
os.write.over(bspFile, createBspConnectionJson(jobs), createFolders = true)
val withDebug = T.log.debugEnabled
if(withDebug) T.log.debug("Enabled debug logging for the BSP server. If you want to disable it, you need to re-run this install command without the --debug option.")
os.write.over(bspFile, createBspConnectionJson(jobs, withDebug), createFolders = true)
}

@deprecated("Use other overload instead.", "Mill after 0.10.7")
def createBspConnectionJson(jobs: Int): String = BSP.createBspConnectionJson(jobs: Int, debug = false)

// creates a Json with the BSP connection details
def createBspConnectionJson(jobs: Int): String = {
def createBspConnectionJson(jobs: Int, debug: Boolean): String = {
// we assume, the classpath is an executable jar here, FIXME
val millPath = sys.props
.get("java.class.path")
@@ -69,8 +74,7 @@ object BSP extends ExternalModule {
"false",
"--jobs",
s"${jobs}"
// s"${BSP.getClass.getCanonicalName.split("[$]").head}/start"
),
) ++ (if(debug) Seq("--debug") else Seq()),
millVersion = BuildInfo.millVersion,
bspVersion = bspProtocolVersion,
languages = languages
2 changes: 1 addition & 1 deletion bsp/src/mill/bsp/BspCompileProblemReporter.scala
Original file line number Diff line number Diff line change
@@ -130,7 +130,6 @@ class BspCompileProblemReporter(
pos.endColumn.orElse(pos.pointer).getOrElse[Int](start.getCharacter.intValue())
)
val diagnostic = new bsp.Diagnostic(new bsp.Range(start, end), problem.message)
diagnostic.setCode(pos.lineContent)
diagnostic.setSource("mill")
diagnostic.setSeverity(
problem.severity match {
@@ -139,6 +138,7 @@ class BspCompileProblemReporter(
case mill.api.Warn => bsp.DiagnosticSeverity.WARNING
}
)
problem.diagnosticCode.foreach { existingCode => diagnostic.setCode(existingCode.code) }
diagnostic
}

6 changes: 4 additions & 2 deletions bsp/src/mill/bsp/MillBspLogger.scala
Original file line number Diff line number Diff line change
@@ -36,7 +36,7 @@ class MillBspLogger(client: BuildClient, taskId: Int, logger: Logger)
client.onBuildTaskProgress(params)
super.ticker(s)
} catch {
case e: Exception =>
case e: Exception => // noop
}
}

@@ -52,7 +52,9 @@ class MillBspLogger(client: BuildClient, taskId: Int, logger: Logger)

override def debug(s: String): Unit = {
super.debug(s)
client.onBuildShowMessage(new ShowMessageParams(MessageType.LOG, s))
if (debugEnabled) {
client.onBuildLogMessage(new LogMessageParams(MessageType.LOG, s))
}
}

}
22 changes: 22 additions & 0 deletions bsp/test/src/BspInstallDebugTests.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package mill.bsp

import mill.util.ScriptTestSuite
import utest._

object BspInstallDebugTests extends ScriptTestSuite(false) {
override def workspaceSlug: String = "bsp-install"
override def scriptSourcePath: os.Path = os.pwd / "bsp" / "test" / "resources" / workspaceSlug

// we purposely enable debugging in this simulated test env
override val debugLog: Boolean = true

def tests: Tests = Tests {
test("BSP install forwards --debug option to server") {
val workspacePath = initWorkspace()
eval("mill.bsp.BSP/install") ==> true
val jsonFile = workspacePath / ".bsp" / s"${BSP.serverName}.json"
os.exists(jsonFile) ==> true
os.read(jsonFile).contains("--debug") ==> true
}
}
}
7 changes: 4 additions & 3 deletions bsp/test/src/BspInstallTests.scala
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
package mill.bsp

import mill.util.ScriptTestSuite
import os._
import utest._

object BspInstallTests extends ScriptTestSuite(false) {
override def workspaceSlug: String = "bsp-install"
override def scriptSourcePath: Path = os.pwd / "bsp" / "test" / "resources" / workspaceSlug
override def scriptSourcePath: os.Path = os.pwd / "bsp" / "test" / "resources" / workspaceSlug

def tests: Tests = Tests {
test("BSP install") {
val workspacePath = initWorkspace()
eval("mill.bsp.BSP/install") ==> true
exists(workspacePath / ".bsp" / s"${BSP.serverName}.json") ==> true
val jsonFile = workspacePath / ".bsp" / s"${BSP.serverName}.json"
os.exists(jsonFile) ==> true
os.read(jsonFile).contains("--debug") ==> false
}
}
}
94 changes: 67 additions & 27 deletions build.sc
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
// plugins and dependencies
import $file.ci.shared
import $file.ci.upload
import $ivy.`org.scalaj::scalaj-http:2.4.2`
import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.1.4`
import $ivy.`com.github.lolgab::mill-mima::0.0.11`
import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.3.0`
import $ivy.`com.github.lolgab::mill-mima::0.0.12`
import $ivy.`net.sourceforge.htmlcleaner:htmlcleaner:2.25`

// imports
import com.github.lolgab.mill.mima
import com.github.lolgab.mill.mima.{
CheckDirection,
@@ -23,7 +26,6 @@ import mill.scalalib._
import mill.scalalib.publish._
import mill.modules.Jvm
import mill.define.SelectMode
import upickle.default.{ReadWriter, macroRW}

object Settings {
val pomOrg = "com.lihaoyi"
@@ -49,9 +51,10 @@ object Settings {
"0.10.4",
"0.10.5",
"0.10.6",
"0.10.7"
"0.10.7",
"0.10.8"
)
val mimaBaseVersions = Seq("0.10.0", "0.10.1", "0.10.2", "0.10.3", "0.10.4")
val mimaBaseVersions = Seq("0.10.0", "0.10.1", "0.10.2", "0.10.3", "0.10.4", "0.10.5", "0.10.6", "0.10.7")
}

object Deps {
@@ -63,7 +66,11 @@ object Deps {

val testScala213Version = "2.13.8"
val testScala212Version = "2.12.6"
val testScala211Version = "2.11.12"
val testScala210Version = "2.10.6"
val testScala30Version = "3.0.2"
val testScala31Version = "3.1.3"
val testScala32Version = "3.2.0"

val testScalaJs06Version = "0.6.33"

@@ -75,21 +82,21 @@ object Deps {

object Scalajs_1 {
val scalajsEnvJsdomNodejs = ivy"org.scala-js::scalajs-env-jsdom-nodejs:1.1.0"
val scalajsEnvNodejs = ivy"org.scala-js::scalajs-env-nodejs:1.3.0"
val scalajsEnvNodejs = ivy"org.scala-js::scalajs-env-nodejs:1.4.0"
val scalajsEnvPhantomjs = ivy"org.scala-js::scalajs-env-phantomjs:1.0.0"
val scalajsSbtTestAdapter = ivy"org.scala-js::scalajs-sbt-test-adapter:1.10.1"
val scalajsLinker = ivy"org.scala-js::scalajs-linker:1.10.1"
}

object Scalanative_0_4 {
val scalanativeTools = ivy"org.scala-native::tools:0.4.5"
val scalanativeUtil = ivy"org.scala-native::util:0.4.5"
val scalanativeNir = ivy"org.scala-native::nir:0.4.5"
val scalanativeTestRunner = ivy"org.scala-native::test-runner:0.4.5"
val scalanativeTools = ivy"org.scala-native::tools:0.4.7"
val scalanativeUtil = ivy"org.scala-native::util:0.4.7"
val scalanativeNir = ivy"org.scala-native::nir:0.4.7"
val scalanativeTestRunner = ivy"org.scala-native::test-runner:0.4.7"
}

val acyclic = ivy"com.lihaoyi::acyclic:0.2.1"
val ammoniteVersion = "2.5.4"
val ammoniteVersion = "2.5.4-33-0af04a5b"
val ammonite = ivy"com.lihaoyi:::ammonite:${ammoniteVersion}"
val ammoniteTerminal = ivy"com.lihaoyi::ammonite-terminal:${ammoniteVersion}"
// Exclude trees here to force the version of we have defined. We use this
@@ -104,7 +111,7 @@ object Deps {

val flywayCore = ivy"org.flywaydb:flyway-core:8.5.13"
val graphvizJava = ivy"guru.nidi:graphviz-java-all-j2v8:0.18.1"
val junixsocket = ivy"com.kohlschutter.junixsocket:junixsocket-core:2.5.1"
val junixsocket = ivy"com.kohlschutter.junixsocket:junixsocket-core:2.5.2"

object jetty {
val version = "8.2.0.v20160908"
@@ -126,18 +133,24 @@ object Deps {
val scalaCheck = ivy"org.scalacheck::scalacheck:1.16.0"
def scalaCompiler(scalaVersion: String) = ivy"org.scala-lang:scala-compiler:${scalaVersion}"
val scalafmtDynamic = ivy"org.scalameta::scalafmt-dynamic:3.5.8"
val scalametaTrees = ivy"org.scalameta::trees:4.5.13"
val scalametaTrees = ivy"org.scalameta::trees:4.6.0"
def scalaReflect(scalaVersion: String) = ivy"org.scala-lang:scala-reflect:${scalaVersion}"
def scalacScoveragePlugin = ivy"org.scoverage:::scalac-scoverage-plugin:1.4.11"
val semanticDB = ivy"org.scalameta:::semanticdb-scalac:4.5.11"
val scoverage2Version = "2.0.5"
def scalacScoverage2Plugin = ivy"org.scoverage:::scalac-scoverage-plugin:${scoverage2Version}"
def scalacScoverage2Reporter = ivy"org.scoverage::scalac-scoverage-reporter:${scoverage2Version}"
def scalacScoverage2Domain = ivy"org.scoverage::scalac-scoverage-domain:${scoverage2Version}"
def scalacScoverage2Serializer = ivy"org.scoverage::scalac-scoverage-serializer:${scoverage2Version}"
val semanticDB = ivy"org.scalameta:::semanticdb-scalac:4.6.0"
val sourcecode = ivy"com.lihaoyi::sourcecode:0.3.0"
val upickle = ivy"com.lihaoyi::upickle:2.0.0"
val utest = ivy"com.lihaoyi::utest:0.7.11"
val windowsAnsi = ivy"io.github.alexarchambault.windows-ansi:windows-ansi:0.0.4"
val zinc = ivy"org.scala-sbt::zinc:1.7.1"
val zinc = ivy"org.scala-sbt::zinc:1.7.2"
val bsp = ivy"ch.epfl.scala:bsp4j:2.1.0-M1"
val fansi = ivy"com.lihaoyi::fansi:0.4.0"
val jarjarabrams = ivy"com.eed3si9n.jarjarabrams::jarjar-abrams-core:1.8.1"
val requests = ivy"com.lihaoyi::requests:0.7.1"
}

def millVersion: T[String] = T { VcsVersion.vcsState().format() }
@@ -210,15 +223,6 @@ trait MillMimaConfig extends mima.Mima {
"mill.api.internal",
"mill.api.experimental"
)

implicit val checkDirectionBackwardUpickleRW: ReadWriter[CheckDirection.Backward.type] = macroRW
implicit val checkDirectionBothUpickleRW: ReadWriter[CheckDirection.Both.type] = macroRW
implicit val checkDirectionForwardUpickleRW: ReadWriter[CheckDirection.Forward.type] = macroRW
implicit val checkDirectionUpickleRW: ReadWriter[CheckDirection] = ReadWriter.merge(
checkDirectionBackwardUpickleRW,
checkDirectionBothUpickleRW,
checkDirectionForwardUpickleRW
)
override def mimaCheckDirection: Target[CheckDirection] = T { CheckDirection.Backward }
override def mimaBinaryIssueFilters: Target[Seq[ProblemFilter]] = T {
issueFilterByModule.getOrElse(this, Seq())
@@ -300,7 +304,11 @@ trait MillScalaModule extends ScalaModule with MillCoursierModule { outer =>
s"-DMILL_SCALA_2_12_VERSION=${Deps.workerScalaVersion212}",
s"-DTEST_SCALA_2_13_VERSION=${Deps.testScala213Version}",
s"-DTEST_SCALA_2_12_VERSION=${Deps.testScala212Version}",
s"-DTEST_SCALA_2_11_VERSION=${Deps.testScala211Version}",
s"-DTEST_SCALA_2_10_VERSION=${Deps.testScala210Version}",
s"-DTEST_SCALA_3_0_VERSION=${Deps.testScala30Version}",
s"-DTEST_SCALA_3_1_VERSION=${Deps.testScala31Version}",
s"-DTEST_SCALA_3_2_VERSION=${Deps.testScala32Version}",
s"-DTEST_UTEST_VERSION=${Deps.utest.dep.version}",
s"-DTEST_SCALAJS_0_6_VERSION=${Deps.testScalaJs06Version}"
) ++ outer.testArgs()
@@ -745,7 +753,9 @@ object contrib extends MillModule {
override def testArgs = T {
val mapping = Map(
"MILL_SCOVERAGE_REPORT_WORKER" -> worker.compile().classes.path,
"MILL_SCOVERAGE_VERSION" -> Deps.scalacScoveragePlugin.dep.version
"MILL_SCOVERAGE2_REPORT_WORKER" -> worker2.compile().classes.path,
"MILL_SCOVERAGE_VERSION" -> Deps.scalacScoveragePlugin.dep.version,
"MILL_SCOVERAGE2_VERSION" -> Deps.scalacScoverage2Plugin.dep.version
)
scalalib.worker.testArgs() ++
scalalib.backgroundwrapper.testArgs() ++
@@ -758,18 +768,36 @@ object contrib extends MillModule {
contrib.buildinfo
)

object worker extends MillApiModule {
object worker extends MillInternalModule {
override def compileModuleDeps = Seq(main.api)
override def moduleDeps = Seq(scoverage.api)
override def compileIvyDeps = T {
Agg(
// compile-time only, need to provide the correct scoverage version runtime
// compile-time only, need to provide the correct scoverage version at runtime
Deps.scalacScoveragePlugin,
// provided by mill runtime
Deps.osLib
)
}
}

object worker2 extends MillInternalModule {
override def compileModuleDeps = Seq(main.api)

override def moduleDeps = Seq(scoverage.api)

override def compileIvyDeps = T {
Agg(
// compile-time only, need to provide the correct scoverage version at runtime
Deps.scalacScoverage2Plugin,
Deps.scalacScoverage2Reporter,
Deps.scalacScoverage2Domain,
Deps.scalacScoverage2Serializer,
// provided by mill runtime
Deps.osLib
)
}
}
}

object buildinfo extends MillModule {
@@ -839,10 +867,12 @@ object contrib extends MillModule {

object artifactory extends MillModule {
override def compileModuleDeps = Seq(scalalib)
override def ivyDeps = T { Agg(Deps.requests) }
}

object codeartifact extends MillModule {
override def compileModuleDeps = Seq(scalalib)
override def ivyDeps = T { Agg(Deps.requests) }
}

object versionfile extends MillModule {
@@ -851,6 +881,16 @@ object contrib extends MillModule {

object bintray extends MillModule {
override def compileModuleDeps = Seq(scalalib)
override def ivyDeps = T { Agg(Deps.requests) }
}

object gitlab extends MillInternalModule with MillAutoTestSetup {
override def compileModuleDeps = Seq(scalalib)
override def ivyDeps = T { Agg(Deps.requests, Deps.osLib) }

override def testModuleDeps: Seq[JavaModule] = super.testModuleDeps ++ Seq(
scalalib
)
}

}
1 change: 1 addition & 0 deletions ci/shared.sc
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@
*/

import $ivy.`org.scalaj::scalaj-http:2.4.2`
import mainargs.main

def unpackZip(zipDest: os.Path, url: String) = {
println(s"Unpacking zip $url into $zipDest")
1 change: 1 addition & 0 deletions ci/upload.sc
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env amm

import scalaj.http._
import mainargs.main

@main
def apply(
11 changes: 11 additions & 0 deletions contrib/gitlab/src/mill/contrib/gitlab/GitlabAuthHeaders.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package mill.contrib.gitlab

case class GitlabAuthHeaders(headers: Seq[(String, String)])

object GitlabAuthHeaders {
def apply(header: String, value: String): GitlabAuthHeaders = GitlabAuthHeaders(Seq(header -> value))

def privateToken(token: String): GitlabAuthHeaders = GitlabAuthHeaders("Private-Token", token)
def deployToken(token: String): GitlabAuthHeaders = GitlabAuthHeaders("Deploy-Token", token)
def jobToken(token: String): GitlabAuthHeaders = GitlabAuthHeaders("Job-Token", token)
}
32 changes: 32 additions & 0 deletions contrib/gitlab/src/mill/contrib/gitlab/GitlabHttpApi.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package mill.contrib.gitlab

import scala.concurrent.duration._

object GitlabUploader {
type Upload = (String, Array[Byte]) => requests.Response
}

class GitlabUploader(
authentication: GitlabAuthHeaders,
readTimeout: Int = 5000,
connectTimeout: Int = 5000
) {
val http = requests.Session(
readTimeout = readTimeout,
connectTimeout = connectTimeout,
maxRedirects = 0,
check = false
)

private val uploadTimeout = 2.minutes.toMillis.toInt

// https://docs.gitlab.com/ee/user/packages/maven_repository/#publish-a-package
def upload(uri: String, data: Array[Byte]): requests.Response = {
http.put(
uri,
readTimeout = uploadTimeout,
headers = authentication.headers,
data = data
)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package mill.contrib.gitlab

import mill.scalalib.publish.Artifact

sealed trait GitlabPackageRepository {
def url(): String
}

// Could also support project name (https://docs.gitlab.com/ee/user/packages/maven_repository/index.html#project-level-maven-endpoint)
// though only ID can be used for publishing.
case class ProjectRepository(baseUrl: String, projectId: Int) extends GitlabPackageRepository {
override def url(): String = baseUrl + s"/api/v4/projects/$projectId/packages/maven"

// https://docs.gitlab.com/ee/api/packages/maven.html#upload-a-package-file
def uploadUrl(artifact: Artifact): String = {
val repoUrl = url()
val group = artifact.group.replace(".", "/")
val id = artifact.id
val version = artifact.version
s"$repoUrl/$group/$id/$version"
}
}

// Note that group repository has some limitations:
// https://docs.gitlab.com/ee/user/packages/maven_repository/#group-level-maven-endpoint
case class GroupRepository(baseUrl: String, groupId: String) extends GitlabPackageRepository {
override def url(): String = baseUrl + s"/api/v4/groups/$groupId/-/packages/maven"
}

// Note that instance level repo has some limitations:
// https://docs.gitlab.com/ee/user/packages/maven_repository/#instance-level-maven-endpoint
case class InstanceRepository(baseUrl: String) extends GitlabPackageRepository {
override def url(): String = baseUrl + s"/api/v4/packages/maven"
}
107 changes: 107 additions & 0 deletions contrib/gitlab/src/mill/contrib/gitlab/GitlabPublishModule.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package mill.contrib.gitlab

import coursier.core.Authentication
import coursier.maven.MavenRepository
import mill._
import mill.api.{Ctx, Logger}
import mill.define.{Command, ExternalModule, Input}
import mill.scalalib.publish.Artifact
import scalalib._

class GitlabAuthenticationException(message: String) extends Exception(message)

trait GitlabPublishModule extends PublishModule {

def publishRepository: ProjectRepository

def skipPublish: Boolean = false

def tokenLookup: GitlabTokenLookup = new GitlabTokenLookup {}

def gitlabHeaders(
log: Logger,
env: Map[String, String],
props: Map[String, String]
): GitlabAuthHeaders = {
val auth = tokenLookup.resolveGitlabToken(log, env, props)
if (auth.isEmpty) {
throw new GitlabAuthenticationException(
s"Unable to resolve authentication with $tokenLookup"
)
}
auth.get
}

def env: Input[Map[String, String]] = T.input(T.ctx().env)
def props: Map[String, String] = sys.props.toMap

def publishGitlab(
readTimeout: Int = 60000,
connectTimeout: Int = 5000
): define.Command[Unit] = T.command {

val gitlabRepo = publishRepository
val gitlabAuth = gitlabHeaders(T.log, env(), props)

val PublishModule.PublishData(artifactInfo, artifacts) = publishArtifacts()
if (skipPublish) {
T.log.info(s"SkipPublish = true, skipping publishing of $artifactInfo")
} else {
val uploader = new GitlabUploader(gitlabAuth)
new GitlabPublisher(
uploader.upload,
gitlabRepo,
readTimeout,
connectTimeout,
T.log
).publish(artifacts.map { case (a, b) => (a.path, b) }, artifactInfo)
}

}
}

object GitlabPublishModule extends ExternalModule {

def publishAll(
personalToken: String,
gitlabRoot: String,
projectId: Int,
publishArtifacts: mill.main.Tasks[PublishModule.PublishData],
readTimeout: Int = 60000,
connectTimeout: Int = 5000
): Command[Unit] = T.command {
val repo = ProjectRepository(gitlabRoot, projectId)
val auth = GitlabAuthHeaders.privateToken(personalToken)

val artifacts: Seq[(Seq[(os.Path, String)], Artifact)] = T.sequence(publishArtifacts.value)().map {
case PublishModule.PublishData(a, s) => (s.map { case (p, f) => (p.path, f) }, a)
}

val uploader = new GitlabUploader(auth)

new GitlabPublisher(
uploader.upload,
repo,
readTimeout,
connectTimeout,
T.log
).publishAll(
artifacts: _*
)
}

implicit def millScoptTargetReads[T] = new mill.main.Tasks.Scopt[T]()

lazy val millDiscover: mill.define.Discover[this.type] = mill.define.Discover[this.type]
}

trait GitlabMavenRepository {
def tokenLookup: GitlabTokenLookup = new GitlabTokenLookup {} // For token discovery
def repository: GitlabPackageRepository // For package discovery

def mavenRepo(implicit context: Ctx): MavenRepository = {
val gitlabAuth = tokenLookup.resolveGitlabToken(context.log, context.env, sys.props.toMap).get
val auth = Authentication(gitlabAuth.headers)
MavenRepository(repository.url(), Some(auth))
}
}
65 changes: 65 additions & 0 deletions contrib/gitlab/src/mill/contrib/gitlab/GitlabPublisher.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package mill.contrib.gitlab

import mill.api.Logger
import mill.scalalib.publish.Artifact
import requests.Response

class GitlabPublisher(
upload: GitlabUploader.Upload,
repo: ProjectRepository,
readTimeout: Int,
connectTimeout: Int,
log: Logger
) {

def publish(fileMapping: Seq[(os.Path, String)], artifact: Artifact): Unit =
publishAll(fileMapping -> artifact)

def publishAll(artifacts: (Seq[(os.Path, String)], Artifact)*): Unit = {
log.info("Publishing artifacts: " + artifacts)

val uploadData = for {
(items, artifact) <- artifacts
files = items.map { case (path, name) => name -> os.read.bytes(path) }
} yield artifact -> files

uploadData
.map { case (artifact, data) =>
publishToRepo(repo, artifact, data)
}
.foreach { case (artifact, result) =>
reportPublishResults(artifact, result)
}
}

private def publishToRepo(
repo: ProjectRepository,
artifact: Artifact,
payloads: Seq[(String, Array[Byte])]
): (Artifact, Seq[Response]) = {
val publishResults = payloads.map { case (fileName, data) =>
log.info(s"Uploading $fileName")
val uploadTarget = repo.uploadUrl(artifact)
val resp = upload(s"$uploadTarget/$fileName", data)
resp
}
artifact -> publishResults
}

private def reportPublishResults(
artifact: Artifact,
publishResults: Seq[requests.Response]
): Unit = {
if (publishResults.forall(_.is2xx)) {
log.info(s"Published $artifact to Gitlab")
} else {
val errors = publishResults.filterNot(_.is2xx).map { response =>
s"Code: ${response.statusCode}, message: ${response.text()}"
}
// Or just log? Fail later?
throw new RuntimeException(
s"Failed to publish $artifact to Gitlab. Errors: \n${errors.mkString("\n")}"
)
}
}
}
106 changes: 106 additions & 0 deletions contrib/gitlab/src/mill/contrib/gitlab/GitlabTokenLookup.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
package mill.contrib.gitlab

import mill.api.Logger

import scala.util.Try

trait GitlabTokenLookup {
import GitlabTokenLookup._

// Default search places for token
def personalTokenEnv: String = "GITLAB_PERSONAL_ACCESS_TOKEN"
def personalTokenProperty: String = "gitlab.personal-access-token"
def personalTokenFile: os.Path = os.home / os.RelPath(".mill/gitlab/personal-access-token")
def personalTokenFileWD: os.Path = os.home / os.RelPath(".gitlab/personal-access-token")
def deployTokenEnv: String = "GITLAB_DEPLOY_TOKEN"
def deployTokenProperty: String = "gitlab.deploy-token"
def deployTokenFile: os.Path = os.home / os.RelPath(".mill/gitlab/deploy-token")
def deployTokenFileWD: os.Path = os.pwd / os.RelPath(".gitlab/deploy-token")
def jobTokenEnv: String = "CI_JOB_TOKEN"

// Default token search order. Implementation picks first found and does not look for the rest.
def tokenSearchOrder: Seq[GitlabToken] = Seq(
Personal(Env(personalTokenEnv)),
Personal(Property(personalTokenProperty)),
Personal(File(personalTokenFile)),
Personal(File(personalTokenFileWD)),
Deploy(Env(deployTokenEnv)),
Deploy(Property(deployTokenProperty)),
Deploy(File(deployTokenFile)),
Deploy(File(deployTokenFileWD)),
CIJob(Env(jobTokenEnv))
)

// Finds gitlab token from this environment. Overriding this is not generally necessary.
def resolveGitlabToken(
log: Logger,
env: Map[String, String],
props: Map[String, String]
): Option[GitlabAuthHeaders] = {
LazyList
.from(tokenSearchOrder)
.map(tt => buildHeaders(tt, env, props))
.tapEach(e => e.left.foreach(msg => log.debug(msg)))
.find(_.isRight)
.flatMap(_.toOption)
}

// Converts GitlabToken to GitlabAuthHeaders. Overriding this is not generally necessary.
def buildHeaders(
token: GitlabToken,
env: Map[String, String],
props: Map[String, String]
): Either[String, GitlabAuthHeaders] = {

def readSource(source: TokenSource): Either[String, String] = source match {
case Env(name) => env.get(name).toRight(s"Could not read environment variable $name")
case Property(prop) => props.get(prop).toRight(s"Could not read system property variable $prop")
case File(path) => Try(os.read(path)).map(_.trim).toEither.left.map(e => s"failed to read file $e")
case Custom(f) => f()
}

token match {
case Personal(source) => readSource(source).map(GitlabAuthHeaders.privateToken)
case Deploy(source) => readSource(source).map(GitlabAuthHeaders.deployToken)
case CIJob(source) => readSource(source).map(GitlabAuthHeaders.jobToken)
case CustomHeader(header, source) => readSource(source).map(GitlabAuthHeaders(header, _))
}
}

override def toString(): String =
s"GitlabEnvironment looking token from ${tokenSearchOrder.mkString(", ")}"
}

object GitlabTokenLookup {

/** Possible types of a Gitlab authentication header.
* - Personal = "Private-Token" ->
* - Deploy = "Deploy-Token"->
* - CIJob = "Job-Token" ->
* - CustomHeader = Use with TokenSource/Custom to produce anything you like
*
* Currently only one custom header is supported. If you need multiple override gitlabToken from GitlabPublishModule
* directly
*/
trait GitlabToken {
def source: TokenSource
}
case class Personal(source: TokenSource) extends GitlabToken
case class Deploy(source: TokenSource) extends GitlabToken
case class CIJob(source: TokenSource) extends GitlabToken
case class CustomHeader(header: String, source: TokenSource) extends GitlabToken

/** Possible source of token value. Either an
* - Env = Environment variable
* - Property = Javas system property
* - File =Contents of a file on local disk.
* - Custom = Own function
*
* Possible additions, that can now be supported with Custom: KeyVault, Yaml, etc..
*/
sealed trait TokenSource
case class Env(name: String) extends TokenSource
case class File(path: os.Path) extends TokenSource
case class Property(property: String) extends TokenSource
case class Custom(f: () => Either[String, String]) extends TokenSource
}
109 changes: 109 additions & 0 deletions contrib/gitlab/test/src/mill/contrib/gitlab/GitlabTests.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
package mill.contrib.gitlab

import mill.util.DummyLogger
import utest.{TestSuite, Tests, assert, assertMatch, test}
import GitlabTokenLookup._
import mill.scalalib.publish.Artifact

import scala.collection.mutable.ListBuffer

object GitlabTests extends TestSuite {
override def tests: Tests = Tests {

object defaultLookup extends GitlabTokenLookup

test("Token search returns first applicable") {
object testLookup extends GitlabTokenLookup {
override def tokenSearchOrder: Seq[GitlabToken] = Seq(
Personal(Property("gl.token1")),
Personal(Property("gl.token2"))
)
}

val none = testLookup.resolveGitlabToken(DummyLogger, Map.empty, Map.empty)
val first = testLookup.resolveGitlabToken(DummyLogger, Map.empty, Map("gl.token1" -> "1"))
val second = testLookup.resolveGitlabToken(DummyLogger, Map.empty, Map("gl.token2" -> "2"))
val both = testLookup.resolveGitlabToken(DummyLogger, Map.empty, Map("gl.token1" -> "1", "gl.token2" -> "2"))

assert(none.isEmpty)
assertMatch(first) { case Some(GitlabAuthHeaders(Seq(("Private-Token", "1")))) => }
assertMatch(second) { case Some(GitlabAuthHeaders(Seq(("Private-Token", "2")))) => }
assertMatch(both) { case Some(GitlabAuthHeaders(Seq(("Private-Token", "1")))) => }
}

test("Token from environment variable") {
val token =
defaultLookup.resolveGitlabToken(DummyLogger, Map("GITLAB_PERSONAL_ACCESS_TOKEN" -> "t"), Map.empty)

assertMatch(token) { case Some(GitlabAuthHeaders(Seq(("Private-Token", "t")))) => }
}

test("Token from property") {
val token = defaultLookup.resolveGitlabToken(
DummyLogger,
Map("GITLAB_DEPLOY_TOKEN" -> "t"),
Map("gitlab.personal-access-token" -> "pt")
)

// personal access token property resolves before deploy token in default lookup
assertMatch(token) { case Some(GitlabAuthHeaders(Seq(("Private-Token", "pt")))) => }
}

test("Token from file") {
val tokenFile = os.pwd / "token.temp"
os.write(tokenFile, "foo")

object fileEnv extends GitlabTokenLookup {
override def tokenSearchOrder: Seq[GitlabToken] = Seq(
Deploy(File(tokenFile))
)
}

val token = fileEnv.resolveGitlabToken(DummyLogger, Map.empty, Map.empty)

os.remove(tokenFile)

assertMatch(token) { case Some(GitlabAuthHeaders(Seq(("Deploy-Token", "foo")))) => }
}

test("Custom token source.") {
object customEnv extends GitlabTokenLookup {
override def tokenSearchOrder: Seq[GitlabToken] = Seq(
Deploy(Custom(() => Right("tok")))
)
}

val token = customEnv.resolveGitlabToken(DummyLogger, Map.empty, Map.empty)

assertMatch(token) { case Some(GitlabAuthHeaders(Seq(("Deploy-Token", "tok")))) => }
}

test("Publish url is correct") {
object uploader extends ((String, Array[Byte]) => requests.Response) {
def apply(url: String, data: Array[Byte]): requests.Response = {
urls.append(url)
requests.Response(url, 200, "Success", new geny.Bytes("".getBytes()), Map.empty, None)
}

val urls: ListBuffer[String] = ListBuffer[String]()
}

val repo = ProjectRepository("https://gitlab.local", 10)
val publisher = new GitlabPublisher(uploader, repo, 1, 1, DummyLogger)

val fakeFile = os.pwd / "dummy.data"
os.write(fakeFile, Array[Byte]())

val artifact = Artifact("test.group", "id", "0.0.0")

publisher.publish(Seq(fakeFile -> "data.file"), artifact)

os.remove(fakeFile)

assert(uploader.urls.size == 1)
assert(
uploader.urls.head == "https://gitlab.local/api/v4/projects/10/packages/maven/test/group/id/0.0.0/data.file"
)
}
}
}
26 changes: 24 additions & 2 deletions contrib/scoverage/api/src/ScoverageReportWorkerApi.scala
Original file line number Diff line number Diff line change
@@ -5,9 +5,31 @@ import mill.api.Ctx
trait ScoverageReportWorkerApi {
import ScoverageReportWorkerApi._

def report(reportType: ReportType, sources: Seq[os.Path], dataDirs: Seq[os.Path])(implicit
@deprecated("Use other overload instead.", "Mill after 0.10.7")
def report(
reportType: ReportType,
sources: Seq[os.Path],
dataDirs: Seq[os.Path]
)(implicit
ctx: Ctx
): Unit
): Unit = {
report(reportType, sources, dataDirs, ctx.workspace)
}

def report(
reportType: ReportType,
sources: Seq[os.Path],
dataDirs: Seq[os.Path],
sourceRoot: os.Path
)(implicit
ctx: Ctx
): Unit = {
// FIXME: We only call the deprecated version here, to preserve binary compatibility. Remove when appropriate.
ctx.log.error(
"Binary compatibility stub may cause infinite loops with StackOverflowError. You need to implement: def report(ReportType, Seq[Path], Seq[Path], os.Path): Unit"
)
report(reportType, sources, dataDirs)
}
}

object ScoverageReportWorkerApi {
142 changes: 127 additions & 15 deletions contrib/scoverage/src/ScoverageModule.scala
Original file line number Diff line number Diff line change
@@ -5,7 +5,9 @@ import mill._
import mill.api.{Loose, PathRef}
import mill.contrib.scoverage.api.ScoverageReportWorkerApi.ReportType
import mill.define.{Command, Persistent, Sources, Target, Task}
import mill.scalalib.api.ZincWorkerUtil
import mill.scalalib.{Dep, DepSyntax, JavaModule, ScalaModule}
import mill.api.Result

/**
* Adds targets to a [[mill.scalalib.ScalaModule]] to create test coverage reports.
@@ -56,49 +58,148 @@ trait ScoverageModule extends ScalaModule { outer: ScalaModule =>
*/
def scoverageVersion: T[String]

private def isScoverage2: Task[Boolean] = T.task { scoverageVersion().startsWith("2.") }

private def isScala3: Task[Boolean] = T.task { ZincWorkerUtil.isScala3(outer.scalaVersion()) }

private def isScala2: Task[Boolean] = T.task { !isScala3() }

/** Binary compatibility shim. */
@deprecated("Use scoverageRuntimeDeps instead.", "Mill after 0.10.7")
def scoverageRuntimeDep: T[Dep] = T {
ivy"org.scoverage::scalac-scoverage-runtime:${outer.scoverageVersion()}"
T.log.error(
"scoverageRuntimeDep is no longer used. To customize your module, use scoverageRuntimeDeps."
)
val result: Result[Dep] = if (isScala3()) {
Result.Failure("When using Scala 3 there is no external runtime dependency")
} else {
scoverageRuntimeDeps().toIndexedSeq.head
}
result
}

def scoverageRuntimeDeps: T[Agg[Dep]] = T {
if (isScala3()) {
Agg.empty
} else {
Agg(ivy"org.scoverage::scalac-scoverage-runtime:${outer.scoverageVersion()}")
}
}

/** Binary compatibility shim. */
@deprecated("Use scoveragePluginDeps instead.", "Mill after 0.10.7")
def scoveragePluginDep: T[Dep] = T {
ivy"org.scoverage:::scalac-scoverage-plugin:${outer.scoverageVersion()}"
T.log.error(
"scoveragePluginDep is no longer used. To customize your module, use scoverageRuntimeDeps."
)
val result: Result[Dep] = if (isScala3()) {
Result.Failure("When using Scala 3 there is no external plugin dependency")
} else {
scoveragePluginDeps().toIndexedSeq.head
}
result
}

def scoveragePluginDeps: T[Agg[Dep]] = T {
val sv = scoverageVersion()
if (isScala3()) {
Agg.empty
} else {
if (isScoverage2()) {
Agg(
ivy"org.scoverage:::scalac-scoverage-plugin:${sv}",
ivy"org.scoverage::scalac-scoverage-domain:${sv}",
ivy"org.scoverage::scalac-scoverage-serializer:${sv}",
ivy"org.scoverage::scalac-scoverage-reporter:${sv}"
)
} else {
Agg(ivy"org.scoverage:::scalac-scoverage-plugin:${sv}")
}
}
}

@deprecated("Use scoverageToolsClasspath instead.", "mill after 0.10.0-M1")
def toolsClasspath: T[Agg[PathRef]] = T {
scoverageToolsClasspath()
}

private def checkVersions = T.task {
val sv = scalaVersion()
val isSov2 = scoverageVersion().startsWith("2.")
(sv.split('.'), isSov2) match {
case (Array("3", "0" | "1", _*), _) => Result.Failure(
"Scala 3.0 and 3.1 is not supported by Scoverage. You have to update to at least Scala 3.2 and Scoverage 2.0"
)
case (Array("3", _*), false) => Result.Failure(
"Scoverage 1.x does not support Scala 3. You have to update to at least Scala 3.2 and Scoverage 2.0"
)
case (Array("2", "11", _*), true) => Result.Failure(
"Scoverage 2.x is not compatible with Scala 2.11. Consider using Scoverage 1.x or switch to a newer Scala version."
)
case _ =>
}
}

def scoverageToolsClasspath: T[Agg[PathRef]] = T {
checkVersions()

scoverageReportWorkerClasspath() ++
resolveDeps(T.task {
Agg(
ivy"org.scoverage:scalac-scoverage-plugin_${mill.BuildInfo.scalaVersion}:${outer.scoverageVersion()}"
// we need to resolve with same Scala version used for Mill, not the project Scala version
val scalaBinVersion = ZincWorkerUtil.scalaBinaryVersion(BuildInfo.scalaVersion)
val sv = scoverageVersion()

val baseDeps = Agg(
ivy"org.scoverage:scalac-scoverage-domain_${scalaBinVersion}:${sv}",
ivy"org.scoverage:scalac-scoverage-serializer_${scalaBinVersion}:${sv}",
ivy"org.scoverage:scalac-scoverage-reporter_${scalaBinVersion}:${sv}"
)

val pluginDep =
Agg(ivy"org.scoverage:scalac-scoverage-plugin_${mill.BuildInfo.scalaVersion}:${sv}")

if (isScala3() && isScoverage2()) {
baseDeps
} else if (isScoverage2()) {
baseDeps ++ pluginDep
} else {
pluginDep
}
})()
}

def scoverageClasspath: T[Agg[PathRef]] = T {
resolveDeps(T.task { Agg(scoveragePluginDep()) })()
resolveDeps(scoveragePluginDeps)()
}

def scoverageReportWorkerClasspath: T[Agg[PathRef]] = T {
val workerKey = "MILL_SCOVERAGE_REPORT_WORKER"
val isScov2 = isScoverage2()

val workerKey =
if (isScov2) "MILL_SCOVERAGE2_REPORT_WORKER"
else "MILL_SCOVERAGE_REPORT_WORKER"

val workerArtifact =
if (isScov2) "mill-contrib-scoverage-worker2"
else "mill-contrib-scoverage-worker"

mill.modules.Util.millProjectModule(
workerKey,
s"mill-contrib-scoverage-worker",
workerArtifact,
repositoriesTask(),
resolveFilter = _.toString.contains("mill-contrib-scoverage-worker")
resolveFilter = _.toString.contains(workerArtifact)
)
}

val scoverage: ScoverageData = new ScoverageData(implicitly)

class ScoverageData(ctx0: mill.define.Ctx) extends Module()(ctx0) with ScalaModule {

def doReport(reportType: ReportType): Task[Unit] = T.task {
ScoverageReportWorker
.scoverageReportWorker()
.bridge(scoverageToolsClasspath().map(_.path))
.report(reportType, allSources().map(_.path), Seq(data().path))
.report(reportType, allSources().map(_.path), Seq(data().path), T.workspace)
}

/**
@@ -121,16 +222,27 @@ trait ScoverageModule extends ScalaModule { outer: ScalaModule =>
override def repositoriesTask: Task[Seq[Repository]] = T.task { outer.repositoriesTask() }
override def compileIvyDeps: Target[Loose.Agg[Dep]] = T { outer.compileIvyDeps() }
override def ivyDeps: Target[Loose.Agg[Dep]] =
T { outer.ivyDeps() ++ Agg(outer.scoverageRuntimeDep()) }
T { outer.ivyDeps() ++ outer.scoverageRuntimeDeps() }
override def unmanagedClasspath: Target[Loose.Agg[PathRef]] = T { outer.unmanagedClasspath() }

/** Add the scoverage scalac plugin. */
override def scalacPluginIvyDeps: Target[Loose.Agg[Dep]] =
T { outer.scalacPluginIvyDeps() ++ Agg(outer.scoveragePluginDep()) }
T { outer.scalacPluginIvyDeps() ++ outer.scoveragePluginDeps() }

/** Add the scoverage specific plugin settings (`dataDir`). */
override def scalacOptions: Target[Seq[String]] =
T { outer.scalacOptions() ++ Seq(s"-P:scoverage:dataDir:${data().path.toIO.getPath()}") }
T {
val extras =
if (isScala3()) {
Seq(s"-coverage-out:${data().path.toIO.getPath()}")
} else {
val base = s"-P:scoverage:dataDir:${data().path.toIO.getPath()}"
if (isScoverage2()) Seq(base, s"-P:scoverage:sourceRoot:${T.workspace}")
else Seq(base)
}

outer.scalacOptions() ++ extras
}

def htmlReport(): Command[Unit] = T.command { doReport(ReportType.Html) }
def xmlReport(): Command[Unit] = T.command { doReport(ReportType.Xml) }
@@ -142,15 +254,15 @@ trait ScoverageModule extends ScalaModule { outer: ScalaModule =>
trait ScoverageTests extends outer.Tests {
override def upstreamAssemblyClasspath = T {
super.upstreamAssemblyClasspath() ++
resolveDeps(T.task { Agg(outer.scoverageRuntimeDep()) })()
resolveDeps(outer.scoverageRuntimeDeps)()
}
override def compileClasspath = T {
super.compileClasspath() ++
resolveDeps(T.task { Agg(outer.scoverageRuntimeDep()) })()
resolveDeps(outer.scoverageRuntimeDeps)()
}
override def runClasspath = T {
super.runClasspath() ++
resolveDeps(T.task { Agg(outer.scoverageRuntimeDep()) })()
resolveDeps(outer.scoverageRuntimeDeps)()
}

// Need the sources compiled with scoverage instrumentation to run.
2 changes: 1 addition & 1 deletion contrib/scoverage/src/ScoverageReport.scala
Original file line number Diff line number Diff line change
@@ -106,7 +106,7 @@ trait ScoverageReport extends Module {
scoverageReportWorkerModule
.scoverageReportWorker()
.bridge(workerModule.scoverageToolsClasspath().map(_.path))
.report(reportType, sourcePaths, dataPaths)
.report(reportType, sourcePaths, dataPaths, T.workspace)
PathRef(T.dest)
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import org.scalatest._
import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpec

class GreetSpec extends WordSpec with Matchers {
class GreetSpec extends AnyWordSpec with Matchers {
"Greet" should {
"work" in {
Greet.greet("Nik", None) shouldBe ("Hello, Nik!")
239 changes: 193 additions & 46 deletions contrib/scoverage/test/src/HelloWorldTests.scala
Original file line number Diff line number Diff line change
@@ -1,22 +1,30 @@
package mill.contrib.scoverage

import mill._
import mill.api.Result
import mill.contrib.buildinfo.BuildInfo
import mill.scalalib.{DepSyntax, ScalaModule, TestModule}
import mill.scalalib.{DepSyntax, SbtModule, ScalaModule, TestModule}
import mill.scalalib.api.ZincWorkerUtil
import mill.util.{TestEvaluator, TestUtil}
import utest._
import utest.framework.TestPath

trait HelloWorldTests extends utest.TestSuite {
def threadCount: Option[Int] = Some(1)

def threadCount: Option[Int]
def testScalaVersion: String

def testScoverageVersion: String
def testScalatestVersion: String

def testScalatestVersion: String = "3.2.13"

def isScala3: Boolean = testScalaVersion.startsWith("3.")
def isScov3: Boolean = testScoverageVersion.startsWith("2.")

val resourcePath = os.pwd / "contrib" / "scoverage" / "test" / "resources" / "hello-world"
val sbtResourcePath = resourcePath / os.up / "hello-world-sbt"
val unmanagedFile = resourcePath / "unmanaged.xml"

trait HelloBase extends TestUtil.BaseModule {
override def millSourcePath = TestUtil.getSrcPathBase() / millOuterCtx.enclosing.split('.')
}
@@ -28,9 +36,13 @@ trait HelloWorldTests extends utest.TestSuite {

object core extends ScoverageModule with BuildInfo {
def scalaVersion = testScalaVersion

def scoverageVersion = testScoverageVersion

override def unmanagedClasspath = Agg(PathRef(unmanagedFile))

override def moduleDeps = Seq(other)

override def buildInfoMembers = T {
Map("scoverageVersion" -> scoverageVersion())
}
@@ -41,29 +53,24 @@ trait HelloWorldTests extends utest.TestSuite {
}
}

object HelloWorldSbt extends HelloBase { outer =>
object core extends ScoverageModule {
object HelloWorldSbt extends HelloBase {
outer =>
object core extends SbtModule with ScoverageModule {
def scalaVersion = testScalaVersion
def scoverageVersion = testScoverageVersion
override def sources = T.sources(
millSourcePath / "src" / "main" / "scala",
millSourcePath / "src" / "main" / "java"
)
override def resources = T.sources { millSourcePath / "src" / "main" / "resources" }

object test extends ScoverageTests with TestModule.ScalaTest {
object test extends SbtModuleTests with ScoverageTests with TestModule.ScalaTest {
override def ivyDeps = Agg(ivy"org.scalatest::scalatest:${testScalatestVersion}")
override def millSourcePath = outer.millSourcePath
override def intellijModulePath = outer.millSourcePath / "src" / "test"
}
}
}

def workspaceTest[T](
m: TestUtil.BaseModule,
resourcePath: os.Path = resourcePath
resourcePath: os.Path = resourcePath,
debugEnabled: Boolean = false
)(t: TestEvaluator => T)(implicit tp: TestPath): T = {
val eval = new TestEvaluator(m, threads = threadCount, debugEnabled = true)
val eval = new TestEvaluator(m, threads = threadCount, debugEnabled = debugEnabled)
os.remove.all(m.millSourcePath)
os.remove.all(eval.outPath)
os.makeDir.all(m.millSourcePath / os.up)
@@ -96,43 +103,86 @@ trait HelloWorldTests extends utest.TestSuite {
val Right((result, evalCount)) =
eval.apply(HelloWorld.core.scoverage.ivyDeps)

val expected = if (isScala3) Agg.empty
else Agg(
ivy"org.scoverage::scalac-scoverage-runtime:${testScoverageVersion}"
)

assert(
result == Agg(ivy"org.scoverage::scalac-scoverage-runtime:${testScoverageVersion}"),
result == expected,
evalCount > 0
)
}
"scalacPluginIvyDeps" - workspaceTest(HelloWorld) { eval =>
val Right((result, evalCount)) =
eval.apply(HelloWorld.core.scoverage.scalacPluginIvyDeps)

val expected = (isScov3, isScala3) match {
case (true, true) => Agg.empty
case (true, false) =>
Agg(
ivy"org.scoverage:::scalac-scoverage-plugin:${testScoverageVersion}",
ivy"org.scoverage::scalac-scoverage-domain:${testScoverageVersion}",
ivy"org.scoverage::scalac-scoverage-serializer:${testScoverageVersion}",
ivy"org.scoverage::scalac-scoverage-reporter:${testScoverageVersion}"
)
case (false, _) =>
Agg(
ivy"org.scoverage:::scalac-scoverage-plugin:${testScoverageVersion}"
)
}
assert(
result == Agg(ivy"org.scoverage:::scalac-scoverage-plugin:${testScoverageVersion}"),
result == expected,
evalCount > 0
)
}
"data" - workspaceTest(HelloWorld) { eval =>
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.data)

val resultPath = result.path.toIO.getPath.replace("""\""", "/")
val expectedEnd =
"mill/target/workspace/mill/contrib/scoverage/HelloWorldTests/eval/HelloWorld/core/scoverage/data/core/scoverage/data.dest"

assert(
result.path.toIO.getPath.replace("""\""", "/").endsWith(
"mill/target/workspace/mill/contrib/scoverage/HelloWorldTests/eval/HelloWorld/core/scoverage/data/core/scoverage/data.dest"
),
resultPath.endsWith(expectedEnd),
evalCount > 0
)
}
"htmlReport" - workspaceTest(HelloWorld) { eval =>
val Right((_, _)) = eval.apply(HelloWorld.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.htmlReport)
assert(evalCount > 0)
val res = eval.apply(HelloWorld.core.scoverage.htmlReport())
if (
res.isLeft && testScalaVersion.startsWith("3.2") && testScoverageVersion.startsWith(
"2."
)
) {
s"""Disabled for Scoverage ${testScoverageVersion} on Scala ${testScalaVersion}, as it fails with "No source root found" message"""
} else {
assert(res.isRight)
val Right((_, evalCount)) = res
assert(evalCount > 0)
""
}
}
"xmlReport" - workspaceTest(HelloWorld) { eval =>
val Right((_, _)) = eval.apply(HelloWorld.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.xmlReport)
assert(evalCount > 0)
val res = eval.apply(HelloWorld.core.scoverage.xmlReport())
if (
res.isLeft && testScalaVersion.startsWith("3.2") && testScoverageVersion.startsWith(
"2."
)
) {
s"""Disabled for Scoverage ${testScoverageVersion} on Scala ${testScalaVersion}, as it fails with "No source root found" message"""
} else {
assert(res.isRight)
val Right((_, evalCount)) = res
assert(evalCount > 0)
""
}
}
"console" - workspaceTest(HelloWorld) { eval =>
val Right((_, _)) = eval.apply(HelloWorld.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.consoleReport)
val Right((_, evalCount)) = eval.apply(HelloWorld.core.scoverage.consoleReport())
assert(evalCount > 0)
}
}
@@ -141,27 +191,55 @@ trait HelloWorldTests extends utest.TestSuite {
val Right((result, evalCount)) =
eval.apply(HelloWorld.core.scoverage.upstreamAssemblyClasspath)

assert(
result.map(_.toString).iterator.exists(_.contains("scalac-scoverage-runtime")),
evalCount > 0
)
val runtimeExistsOnClasspath =
result.map(_.toString).iterator.exists(_.contains("scalac-scoverage-runtime"))
if (isScala3) {
assert(
!runtimeExistsOnClasspath,
evalCount > 0
)
} else {
assert(
runtimeExistsOnClasspath,
evalCount > 0
)
}
}
"compileClasspath" - workspaceTest(HelloWorld) { eval =>
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.compileClasspath)

assert(
result.map(_.toString).iterator.exists(_.contains("scalac-scoverage-runtime")),
evalCount > 0
)
val runtimeExistsOnClasspath =
result.map(_.toString).iterator.exists(_.contains("scalac-scoverage-runtime"))
if (isScala3) {
assert(
!runtimeExistsOnClasspath,
evalCount > 0
)
} else {
assert(
runtimeExistsOnClasspath,
evalCount > 0
)
}
}
// TODO: document why we disable for Java9+
"runClasspath" - workspaceTest(HelloWorld) { eval =>
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scoverage.runClasspath)

assert(
result.map(_.toString).exists(_.contains("scalac-scoverage-runtime")),
evalCount > 0
)
val runtimeExistsOnClasspath =
result.map(_.toString).iterator.exists(_.contains("scalac-scoverage-runtime"))

if (isScala3) {
assert(
!runtimeExistsOnClasspath,
evalCount > 0
)
} else {
assert(
runtimeExistsOnClasspath,
evalCount > 0
)
}
}
}
}
@@ -170,34 +248,103 @@ trait HelloWorldTests extends utest.TestSuite {
"scoverage" - {
"htmlReport" - workspaceTest(HelloWorldSbt, sbtResourcePath) { eval =>
val Right((_, _)) = eval.apply(HelloWorldSbt.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.htmlReport)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.htmlReport())
assert(evalCount > 0)
}
"xmlReport" - workspaceTest(HelloWorldSbt, sbtResourcePath) { eval =>
val Right((_, _)) = eval.apply(HelloWorldSbt.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.xmlReport)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.xmlReport())
assert(evalCount > 0)
}
"console" - workspaceTest(HelloWorldSbt, sbtResourcePath) { eval =>
val Right((_, _)) = eval.apply(HelloWorldSbt.core.test.compile)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.consoleReport)
val Right((result, evalCount)) = eval.apply(HelloWorldSbt.core.scoverage.consoleReport())
assert(evalCount > 0)
}
}
}
}
}

object HelloWorldTests_2_12 extends HelloWorldTests {
override def threadCount = Some(1)
trait FailedWorldTests extends HelloWorldTests {
def errorMsg: String
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE2_VERSION", ???)

override def tests: Tests = utest.Tests {
"HelloWorld" - {
val mod = HelloWorld
"shouldFail" - {
"scoverageToolsCp" - workspaceTest(mod) { eval =>
val Left(Result.Failure(msg, _)) = eval.apply(mod.core.scoverageToolsClasspath)
assert(msg == errorMsg)
}
"other" - workspaceTest(mod) { eval =>
val Left(Result.Failure(msg, _)) = eval.apply(mod.core.scoverage.xmlReport())
assert(msg == errorMsg)
}
}
}
"HelloWorldSbt" - {
val mod = HelloWorldSbt
"shouldFail" - {
"scoverageToolsCp" - workspaceTest(mod) { eval =>
val res = eval.apply(mod.core.scoverageToolsClasspath)
assert(res.isLeft)
println(s"res: ${res}")
val Left(Result.Failure(msg, _)) = res
assert(msg == errorMsg)
}
"other" - workspaceTest(mod) { eval =>
val Left(Result.Failure(msg, _)) = eval.apply(mod.core.scoverage.xmlReport())
assert(msg == errorMsg)
}
}
}
}
}

object Scoverage1Tests_2_12 extends HelloWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("MILL_SCALA_2_12_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE_VERSION", ???)
override def testScalatestVersion = "3.0.8"
}

object HelloWorldTests_2_13 extends HelloWorldTests {
override def threadCount = Some(1)
object Scoverage1Tests_2_13 extends HelloWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_2_13_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE_VERSION", ???)
}

object Scoverage2Tests_2_13 extends HelloWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_2_13_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE2_VERSION", ???)
}

object Scoverage2Tests_3_2 extends HelloWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_3_2_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE2_VERSION", ???)
}

object Scoverage1Tests_3_0 extends FailedWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_3_0_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE_VERSION", ???)
override def testScalatestVersion = "3.0.8"
override val errorMsg =
"Scala 3.0 and 3.1 is not supported by Scoverage. You have to update to at least Scala 3.2 and Scoverage 2.0"
}

object Scoverage1Tests_3_2 extends FailedWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_3_2_VERSION", ???)
override def testScoverageVersion = sys.props.getOrElse("MILL_SCOVERAGE_VERSION", ???)
override val errorMsg =
"Scoverage 1.x does not support Scala 3. You have to update to at least Scala 3.2 and Scoverage 2.0"
}

object Scoverage2Tests_2_11 extends FailedWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_2_11_VERSION", ???)
override val errorMsg =
"Scoverage 2.x is not compatible with Scala 2.11. Consider using Scoverage 1.x or switch to a newer Scala version."
}

object Scoverage2Tests_3_1 extends FailedWorldTests {
override def testScalaVersion: String = sys.props.getOrElse("TEST_SCALA_3_1_VERSION", ???)
override val errorMsg =
"Scala 3.0 and 3.1 is not supported by Scoverage. You have to update to at least Scala 3.2 and Scoverage 2.0"
}
9 changes: 7 additions & 2 deletions contrib/scoverage/worker/src/ScoverageReportWorkerImpl.scala
Original file line number Diff line number Diff line change
@@ -5,12 +5,17 @@ import _root_.scoverage.report.{CoverageAggregator, ScoverageHtmlWriter, Scovera
import mill.api.Ctx
import mill.contrib.scoverage.api.ScoverageReportWorkerApi.ReportType

/**
* Scoverage Worker for Scoverage 1.x
*/
class ScoverageReportWorkerImpl extends ScoverageReportWorkerApi {

override def report(
reportType: ReportType,
sources: Seq[os.Path],
dataDirs: Seq[os.Path]
dataDirs: Seq[os.Path],
// ignored in Scoverage 1.x
sourceRoot: os.Path
)(implicit ctx: Ctx): Unit =
try {
ctx.log.info(s"Processing coverage data for ${dataDirs.size} data locations")
@@ -34,7 +39,7 @@ class ScoverageReportWorkerImpl extends ScoverageReportWorkerApi {
ctx.log.error(s"No coverage data found in [${dataDirs.mkString(", ")}]")
}
} catch {
case e =>
case e: Throwable =>
ctx.log.error(s"Exception while building coverage report. ${e.getMessage()}")
e.printStackTrace()
throw e
46 changes: 46 additions & 0 deletions contrib/scoverage/worker2/src/ScoverageReportWorkerImpl.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package mill.contrib.scoverage.worker

import mill.contrib.scoverage.api.ScoverageReportWorkerApi
import _root_.scoverage.reporter.{CoverageAggregator, ScoverageHtmlWriter, ScoverageXmlWriter}
import mill.api.Ctx
import mill.contrib.scoverage.api.ScoverageReportWorkerApi.ReportType

/**
* Scoverage Worker for Scoverage 2.x
*/
class ScoverageReportWorkerImpl extends ScoverageReportWorkerApi {

override def report(
reportType: ReportType,
sources: Seq[os.Path],
dataDirs: Seq[os.Path],
sourceRoot: os.Path
)(implicit ctx: Ctx): Unit =
try {
ctx.log.info(s"Processing coverage data for ${dataDirs.size} data locations")
CoverageAggregator.aggregate(dataDirs.map(_.toIO), sourceRoot.toIO) match {
case Some(coverage) =>
val sourceFolders = sources.map(_.toIO)
val folder = ctx.dest
os.makeDir.all(folder)
reportType match {
case ReportType.Html =>
new ScoverageHtmlWriter(sourceFolders, folder.toIO, None)
.write(coverage)
case ReportType.Xml =>
new ScoverageXmlWriter(sourceFolders, folder.toIO, false, None)
.write(coverage)
case ReportType.Console =>
ctx.log.info(s"Statement coverage.: ${coverage.statementCoverageFormatted}%")
ctx.log.info(s"Branch coverage....: ${coverage.branchCoverageFormatted}%")
}
case None =>
ctx.log.error(s"No coverage data found in [${dataDirs.mkString(", ")}]")
}
} catch {
case e: Throwable =>
ctx.log.error(s"Exception while building coverage report. ${e.getMessage()}")
e.printStackTrace()
throw e
}
}
10 changes: 5 additions & 5 deletions docs/antora/antora.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
name: mill
title: Mill Documentation
version: '0.10.7'
version: '0.10.8'
nav:
- modules/ROOT/nav.adoc
asciidoc:
attributes:
mill-version: '0.10.7'
mill-last-tag: '0.10.7'
mill-version: '0.10.8'
mill-last-tag: '0.10.8'
bsp-version: '2.0.0'
example-semanticdb-version: '4.5.11'
example-scala-version: '2.13.8'
example-semanticdb-version: '4.6.0'
example-scala-version: '2.13.10'
1 change: 1 addition & 0 deletions docs/antora/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -17,6 +17,7 @@
** xref:Plugin_Codeartifact.adoc[]
** xref:Plugin_Docker.adoc[]
** xref:Plugin_Flyway.adoc[]
** xref:Plugin_Gitlab.adoc[]
** xref:Plugin_Play.adoc[]
** xref:Plugin_Proguard.adoc[]
** xref:Plugin_ScalaPB.adoc[]
2 changes: 1 addition & 1 deletion docs/antora/modules/ROOT/pages/Common_Project_Layouts.adoc
Original file line number Diff line number Diff line change
@@ -120,7 +120,7 @@ If you are following the https://www.scala-js.org/doc/tutorial/basic/[Scala.js T
.`build.sc`
[source,scala]
----
import mill._, scalalib._, scalanativelib._
import mill._, scalalib._, scalanativelib._, mill.scalanativelib.api._
object hello extends ScalaNativeModule {
def scalaVersion = "2.11.12"
2 changes: 1 addition & 1 deletion docs/antora/modules/ROOT/pages/Configuring_Mill.adoc
Original file line number Diff line number Diff line change
@@ -423,7 +423,7 @@ Your project structure for this would look something like this:
----

After generating your docs with `mill example.docJar` you'll find by opening
your `out/app/docJar.dest/javadoc/index.hmtl` locally in your browser you'll
your `out/app/docJar.dest/javadoc/index.html` locally in your browser you'll
have a full static site including your API docs, your blog, and your
documenation!

1 change: 1 addition & 0 deletions docs/antora/modules/ROOT/pages/Contrib_Plugins.adoc
Original file line number Diff line number Diff line change
@@ -37,6 +37,7 @@ import $ivy.`com.lihaoyi::mill-contrib-bloop:`
* xref:Plugin_Codeartifact.adoc[]
* xref:Plugin_Docker.adoc[]
* xref:Plugin_Flyway.adoc[]
* xref:Plugin_Gitlab.adoc[]
* xref:Plugin_Play.adoc[]
* xref:Plugin_Proguard.adoc[]
* xref:Plugin_ScalaPB.adoc[]
220 changes: 220 additions & 0 deletions docs/antora/modules/ROOT/pages/Plugin_Gitlab.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
= Gitlab

This plugin provides publishing and dependencies to Gitlab package registries.

Gitlab does not support http basic auth so using PublishModule, artifactory-
or bintray-plugin does not work. This plugin tries to provide as automatic
as possible support for gitlab package registries and automatic detection of
gitlab CI/CD pipeline.

== Publishing

Most trivial publish config is:

.`build.sc`
[source,scala]
----
import mill._, scalalib._, mill.scalalib.publish._
import $ivy.`com.lihaoyi::mill-contrib-gitlab:`
import mill.contrib.gitlab._
object lib extends ScalaModule with GitlabPublishModule {
// PublishModule requirements:
override def publishVersion = "0.0.1"
override def pomSettings = ??? // PomSettings(...)
// GitlabPublishModule requirements
// 42 is the project id in your gitlab
override def publishRepository = ProjectRepository("https://gitlab.local", 42)
}
----

`publishVersion` and `pomSettings` come from `PublishModule`. `GitlabPublishModule`
requires you to
set `publishRepository` for target of artifact publishing. Note that this *must* be a
project repository defined by project id (publishing to other type of repositories is not
https://docs.gitlab.com/ee/user/packages/maven_repository/#use-the-gitlab-endpoint-for-maven-packages[supported]).

You can also override `def gitlabTokenLookup: GitlabTokenLookup` if default token lookup
does suit your needs. Configuring lookup is documented <<Configuring token lookup,below>>.

=== Default token lookup

By default, plugin first tries to look for
personal access token, then deploy token and lastly ci job token. Default search order is

. Environment variable `GITLAB_PERSONAL_ACCESS_TOKEN`
. System property `gitlab.personal-access-token`
. File `~/.mill/gitlab/personal-access-token`
. File `.gitlab/personal-access-token`
. Environment variable `GITLAB_DEPLOY_TOKEN`
. System property `gitlab.deploy-token`
. File `~/.mill/gitlab/deploy-token`
. File `.gitlab/deploy-token`
. Environment variable `CI_JOB_TOKEN`

Items 1-4 are personal access tokens, 5-8 deploy tokens and 9 is job token.

Because contents of `$CI_JOB_TOKEN` is checked publishing should just work when run in Gitlab
CI/CD pipeline. If you want something else than default lookup configuration can be
overridden. There are different ways of configuring token resolving.

=== Configuring token lookup

==== Override search places

If you want to change environment variable names, property names of paths where plugin looks
for token. It can be done by overriding their respective values in `GitlabTokenLookup`. For
example:

.`build.sc`
[source,scala]
----
override def tokenLookup: GitlabTokenLookup = new GitlabTokenLookup {
override def personalTokenEnv = "MY_TOKEN"
override def deployTokenFile: os.Path = os.root/"etc"/"tokens"/"gitlab-deploy-token"
}
----

This still keeps the default search order, but allows changes to places where to look from.


==== Add or change tokenSearchOrder

If original search order is too wide, or you would like to add places to look you can
override the `tokenSearchOrder`. Example below ignores default search order and adds five
places to search from.

.`build.sc`
[source,scala]
----
override tokenLookup: GitlabTokenLookup = new GitlabTokenLookup {
// Just to add to default sequence set: super.tokenSearchOrder ++ Seq(...
override def tokenSearchOrder: Seq[GitlabToken] = Seq(
Personal(Env("MY_PRIVATE_TOKEN")),
Personal(Property("gitlab.private-token")),
Deploy(File(os.root/"etc"/"gitlab-deploy-token")),
Deploy(Custom(myCustomTokenSource)),
CustomHeader("my-header", Custom(myCustomTokenSource))
)
def myCustomTokenSource(): Either[String, String] = Right("foo")
}
----

There are two things happening here. First Gitlab needs right kind of token for right
header.`Personal` creates "Private-Token" header, `Deploy` produces "Deploy-Token" and
`CIJob` creates "Job-Token". Finally, any custom header can be set with `CustomHeader`.

Secondly after token type plugin needs information where to load token from. There are
four possibilities

1. `Env`: From environment variable
2. `Property`: From system property
3. `File`: From file (content is trimmed, usually at least \n at the end is present)
4. `Custom`: Any `() => Either[String, String]` function

=== Override search logic completely

Modifying the lookup order with `Custom` should be powerful enough but if really
necessary one can also override GitlabPublishModules `gitlabHeaders`.
If for some reason you need to set multiple headers this is currently the only way.

[source,scala]
----
object myModule extends ScalaModule with GitlabPublishModule {
override def gitlabHeaders(
log: Logger,
env: Map[String, String], // Environment variables
props: Map[String, String] // System properties
): GitlabAuthHeaders = {
// This uses default lookup and ads custom headers
val access = tokenLookup.resolveGitlabToken(log, env, props)
val accessHeader = access.fold(Seq.empty[(String, String)])(_.headers)
GitlabAuthHeaders(
accessHeader ++ Seq(
"header1" -> "value1",
"header2" -> "value2"
))
}
}
----

=== Other

For convenience GitlabPublishModule has `def skipPublish: Boolean` that defaults to `false`.
This allows running CI/CD pipeline and skip publishing (for example if you
are not ready increase version number just yet).

== Gitlab package registry dependency

Making mill to fetch package from gitlab package repository is simple:

[source,scala]
----
// DON'T DO THIS
def repositoriesTask = T.task {
super.repositoriesTask() ++ Seq(
MavenRepository("https://gitlab.local/api/v4/projects/42/packages/maven",
Some(Authentication(Seq(("Private-Token", "<<private-token>>"))))))
}
----

However, **we do not want to expose secrets in our build configuration**.
We would like to use the same authentication mechanisms when publishing. This extension
provides trait `GitlabMavenRepository` to ease that.

[source,scala]
----
object myPackageRepository extends GitlabMavenRepository {
// Customize if needed, omit if unnecessary
// override def tokenLookup: GitlabTokenLookup = new GitlabTokenLookup {}
// Needed. Can also be ProjectRepository or InstanceRepository
def repository = GroupRepository("https://gitlab.local", "MY_GITLAB_GROUP")
}
//
object myModule extends ScalaModule {
// ...
def repositoriesTask = T.task {
super.repositoriesTask() ++ Seq(myPackageRepository.mavenRepo())
}
}
----

`GitlabMavenRepository` has overridable `def tokenLookup: GitlabTokenLookup` and you can use
the same configuration mechanisms as described <<Configuring token lookup,above>>.

_Why the intermediate `packageRepository` object?_

Nothing actually prevents you from implementing `GitlabMavenRepository` trait. Having
a separate object makes configuration more sharable when you have multiple registries.

Gitlab supports instance, group and project registries. When depending on
multiple private packages is more convenient to depend on instance or
group level registry. However, publishing is only possible to project registry
and that is why `GitlabPublishModule` requires a `GitlabProjectRepository` instance.

== Future development / caveats

* Some maven / gitlab feature I'm missing?
* Tuning GitlabMavenRepository.
** Support multiple registries?
** Prefer implemented trait to repo object in documentation?
* More configuration, timeouts etc
* Some other common token source / type I've overlooked
* Container registry support with docker module
* Other Gitlab auth methods? (deploy keys?, ...)
* Tested with Gitlab 15.2.2. Older versions might not work


== References

* Mill contrib https://github.com/com-lihaoyi/mill/tree/main/contrib/artifactory/src/mill/contrib/artifactory[artifactory]
and https://github.com/com-lihaoyi/mill/tree/main/contrib/bintray/src/mill/contrib/bintray[bintray]
modules source code
* https://github.com/azolotko/sbt-gitlab[sbt-gitlab]
* Gitlab documentation
** https://docs.gitlab.com/ee/user/packages/maven_repository/index.html[maven package registry]
** https://docs.gitlab.com/ee/api/packages/maven.html[Gitlab maven api]
72 changes: 72 additions & 0 deletions docs/antora/modules/ROOT/pages/Thirdparty_Plugins.adoc
Original file line number Diff line number Diff line change
@@ -71,6 +71,40 @@ Limited bash completion support.

Project home: https://github.com/lefou/mill-bash-completion

== CI Release

`mill-ci-release` is a wrapper around the existing publish functionality of
Mill with the aim to making releasing your project in GitHub Actions to Maven
easier by automating common setup such as setting up gpg in CI, setting up
versioning, and ensuring merges to into your main branch get published as a
SNAPSHOT. If you're coming from sbt, then you're likely familiar with
https://github.com/sbt/sbt-ci-release[`sbt-ci-release`] which this plugin
imitates.

Project home: https://github.com/ckipp01/mill-ci-release

=== Quickstart

To get started, you'll want to use `CiReleaseModule` as a drop in replacement
where you'd normally use the Mill `PublishModule` and then ensure you implement
everything that `PublishModule` requires.

Secondly, you'll need to ensure you have a few environment variables correctly
set in your GitHub repo. You can see detailed instuctions on which are
necessary https://github.com/ckipp01/mill-ci-release#secrets[here].

Then in CI to publish you'll simply issue a single command:

[source,yaml]
----
- run: mill -i io.kipp.mill.ci.release.ReleaseModule/publishAll
----

This will automatically grab all the artifacts that you've defined to publish
in your build and publish them. Your version will automatically be managed by
https://github.com/lefou/mill-vcs-version[`mill-vcs-version`] and if your
version ends in `-SNAPSHOT` you're project will be published to Sonatype
Snapshots or to the normal releases if it's a new tag.

== DGraph

@@ -716,6 +750,44 @@ object project extends ScalaModule with ScalafixModule {
project.fix A Scalafix linter error was reported
----

== SCIP (SCIP Code Intelligence Protocol)

Support for generating https://about.sourcegraph.com/blog/announcing-scip[SCIP]
indexes from your Mill build. This is most commonly used to power intelligent
code navigation on https://sourcegraph.com/[Sourcegraph].

Project home: https://github.com/ckipp01/mill-scip

=== Quickstart

The recommended way to use `mill-scip` is via the
https://sourcegraph.github.io/scip-java/[`scip-java`] cli tool that can be
installed via https://get-coursier.io/[Coursier].

[source, shell script]
----
cs install scip-java
----

Once you have `scip-java` installed the following command and the root of your
Mill build will generate an index and place it at the root of your project.

[source, shell script]
----
scip-java index
----

You can also manually trigger this with Mill by doing the following:

[source, shell script]
----
mill --import ivy:io.chris-kipp::mill-scip::0.2.2 io.kipp.mill.scip.Scip/generate
----

This will then generate your `index.scip` inside of
`out/io/kipp/mill/scip/Scip/generate.dest/`.

== Shell Completions

As Mill is a tool often used from the CLI (Command line interface), you may be also interested in installing some completion support for your preferred shell:
Binary file added index.scip
Binary file not shown.
12 changes: 12 additions & 0 deletions main/api/src/mill/api/CompileProblemReporter.scala
Original file line number Diff line number Diff line change
@@ -30,6 +30,18 @@ trait Problem {
def message: String

def position: ProblemPosition

// TODO Remove default implementation in 0.11.x series
def diagnosticCode: Option[DiagnosticCode] = None
}

/**
* Unique diagnostic code given from the compiler with an optional further explanation.
*/
trait DiagnosticCode {
def code: String

def explanation: Option[String]
}

/**
2 changes: 1 addition & 1 deletion main/api/src/mill/api/Logger.scala
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package mill.api

import java.io._
import java.io.{InputStream, PrintStream}

/**
* The standard logging interface of the Mill build tool.
2 changes: 1 addition & 1 deletion main/core/src/mill/define/ParseArgs.scala
Original file line number Diff line number Diff line change
@@ -49,7 +49,7 @@ object ParseArgs {
*/
@tailrec
def separated(result: Seq[Seq[String]], rest: Seq[String]): Seq[Seq[String]] = rest match {
case Seq() => result
case Seq() => if (result.nonEmpty) result else Seq(Seq())
case r =>
val (next, r2) = r.span(_ != TargetSeparator)
separated(
1 change: 1 addition & 0 deletions main/core/src/mill/eval/Evaluator.scala
Original file line number Diff line number Diff line change
@@ -669,6 +669,7 @@ class Evaluator private[Evaluator] (
case Some(path) => MultiLogger(
logger.colored,
logger,
// we always enable debug here, to get some more context in log files
new FileLogger(logger.colored, path, debugEnabled = true),
logger.inStream
)
28 changes: 28 additions & 0 deletions main/src/mill/main/LevenshteinDistance.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package mill.main

/**
* Compute the Levenshtein Distance.
*/
// Using as trait to keep binary compatibility within Mill 0.10
// TODO: make it an object in Mill 0.11
trait LevenshteinDistance {
def minimum(i1: Int, i2: Int, i3: Int) = math.min(math.min(i1, i2), i3)

/**
* Short Levenshtein distance algorithm, based on
*
* https://rosettacode.org/wiki/Levenshtein_distance#Scala
*/
def editDistance(s1: String, s2: String) = {
val dist = Array.tabulate(s2.length + 1, s1.length + 1) { (j, i) =>
if (j == 0) i else if (i == 0) j else 0
}

for (j <- 1 to s2.length; i <- 1 to s1.length)
dist(j)(i) =
if (s2(j - 1) == s1(i - 1)) dist(j - 1)(i - 1)
else minimum(dist(j - 1)(i) + 1, dist(j)(i - 1) + 1, dist(j - 1)(i - 1) + 1)

dist(s2.length)(s1.length)
}
}
318 changes: 48 additions & 270 deletions main/src/mill/main/Resolve.scala
Original file line number Diff line number Diff line change
@@ -2,254 +2,13 @@ package mill.main

import mill.define._
import mill.define.TaskModule
import ammonite.util.Res
import mainargs.{MainData, TokenGrouping}
import mill.main.ResolveMetadata.singleModuleMeta

import scala.collection.immutable
import scala.reflect.ClassTag

object ResolveMetadata extends Resolve[String] {
def singleModuleMeta(obj: Module, discover: Discover[_], isRootModule: Boolean): Seq[String] = {
val modules = obj.millModuleDirectChildren.map(_.toString)
val targets =
obj
.millInternal
.reflectAll[NamedTask[_]]
.map(_.toString)
val commands =
for {
(cls, entryPoints) <- discover.value
if cls.isAssignableFrom(obj.getClass)
ep <- entryPoints
} yield
if (isRootModule) ep._2.name
else s"$obj.${ep._2.name}"

modules ++ targets ++ commands
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[String]] = {
def direct = singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty)
last match {
case "__" =>
Right(
// Filter out our own module in
obj.millInternal.modules.flatMap(m => singleModuleMeta(m, discover, m == obj))
)
case "_" => Right(direct)
case _ =>
direct.find(_.split('.').last == last) match {
case None =>
Resolve.errorMsgLabel(direct, Seq(Segment.Label(last)), obj.millModuleSegments.value)
case Some(s) => Right(Seq(s))
}
}
}

def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, List[String]] = {
obj match {
case c: Cross[Module] =>
last match {
case List("__") => Right(c.items.map(_._2.toString))
case items =>
c.items
.filter(_._1.length == items.length)
.filter(_._1.zip(last).forall { case (a, b) => b == "_" || a.toString == b })
.map(_._2.toString) match {
case Nil =>
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
last,
obj.millModuleSegments.value
)
case res => Right(res)
}

}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}
}

object ResolveSegments extends Resolve[Segments] {

override def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[Segments]] = {
obj match {
case c: Cross[Module] =>
last match {
case List("__") => Right(c.items.map(_._2.millModuleSegments))
case items =>
c.items
.filter(_._1.length == items.length)
.filter(_._1.zip(last).forall { case (a, b) => b == "_" || a.toString == b })
.map(_._2.millModuleSegments) match {
case Nil =>
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
last,
obj.millModuleSegments.value
)
case res => Right(res)
}
}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[Segments]] = {
val target =
obj
.millInternal
.reflectSingle[Target[_]](last)
.map(t => Right(t.ctx.segments))

val command =
Resolve
.invokeCommand(obj, last, discover.asInstanceOf[Discover[Module]], rest)
.headOption
.map(_.map(_.ctx.segments))

val module =
obj.millInternal
.reflectNestedObjects[Module]
.find(_.millOuterCtx.segment == Segment.Label(last))
.map(m => Right(m.millModuleSegments))

command orElse target orElse module match {
case None =>
Resolve.errorMsgLabel(
singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty),
Seq(Segment.Label(last)),
obj.millModuleSegments.value
)

case Some(either) => either.right.map(Seq(_))
}
}
}

object ResolveTasks extends Resolve[NamedTask[Any]] {

def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[NamedTask[Any]]] = {
obj match {
case c: Cross[Module] =>
Resolve.runDefault(obj, Segment.Cross(last), discover, rest).flatten.headOption match {
case None =>
Left(
"Cannot find default task to evaluate for module " +
Segments((Segment.Cross(last) +: obj.millModuleSegments.value).reverse: _*).render
)
case Some(v) => v.map(Seq(_))
}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[NamedTask[Any]]] = last match {
case "__" =>
Right(
obj.millInternal.modules
.filter(_ != obj)
.flatMap(m => m.millInternal.reflectAll[NamedTask[_]])
)
case "_" => Right(obj.millInternal.reflectAll[NamedTask[_]])

case _ =>
val target =
obj
.millInternal
.reflectSingle[NamedTask[_]](last)
.map(Right(_))

val command = Resolve.invokeCommand(
obj,
last,
discover.asInstanceOf[Discover[Module]],
rest
).headOption

command orElse target orElse Resolve.runDefault(
obj,
Segment.Label(last),
discover,
rest
).flatten.headOption match {
case None =>
Resolve.errorMsgLabel(
singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty),
Seq(Segment.Label(last)),
obj.millModuleSegments.value
)

// Contents of `either` *must* be a `Task`, because we only select
// methods returning `Task` in the discovery process
case Some(either) => either.map(Seq(_))
}
}
}

object Resolve {
def minimum(i1: Int, i2: Int, i3: Int) = math.min(math.min(i1, i2), i3)

/**
* Short Levenshtein distance algorithm, based on
*
* https://rosettacode.org/wiki/Levenshtein_distance#Scala
*/
def editDistance(s1: String, s2: String) = {
val dist = Array.tabulate(s2.length + 1, s1.length + 1) { (j, i) =>
if (j == 0) i else if (i == 0) j else 0
}

for (j <- 1 to s2.length; i <- 1 to s1.length)
dist(j)(i) =
if (s2(j - 1) == s1(i - 1)) dist(j - 1)(i - 1)
else minimum(dist(j - 1)(i) + 1, dist(j)(i - 1) + 1, dist(j - 1)(i - 1) + 1)

dist(s2.length)(s1.length)
}
object Resolve extends LevenshteinDistance {

def unableToResolve(last: Segment, revSelectorsSoFar: Seq[Segment]): String = {
unableToResolve(Segments((last +: revSelectorsSoFar).reverse: _*).render)
@@ -334,7 +93,12 @@ object Resolve {
)
}

def invokeCommand(target: Module, name: String, discover: Discover[Module], rest: Seq[String]) =
def invokeCommand(
target: Module,
name: String,
discover: Discover[Module],
rest: Seq[String]
): immutable.Iterable[Either[String, Command[_]]] =
for {
(cls, entryPoints) <- discover.value
if cls.isAssignableFrom(target.getClass)
@@ -370,24 +134,31 @@ object Resolve {
}
}

def runDefault(obj: Module, last: Segment, discover: Discover[_], rest: Seq[String]) = for {
child <- obj.millInternal.reflectNestedObjects[Module]
if child.millOuterCtx.segment == last
res <- child match {
case taskMod: TaskModule =>
Some(
invokeCommand(
child,
taskMod.defaultCommandName(),
discover.asInstanceOf[Discover[Module]],
rest
).headOption
)
case _ => None
}
} yield res

def runDefault(
obj: Module,
last: Segment,
discover: Discover[_],
rest: Seq[String]
): Array[Option[Either[String, Command[_]]]] = {
for {
child <- obj.millModuleDirectChildren
if child.millOuterCtx.segment == last
res <- child match {
case taskMod: TaskModule =>
Some(
invokeCommand(
child,
taskMod.defaultCommandName(),
discover.asInstanceOf[Discover[Module]],
rest
).headOption
)
case _ => None
}
} yield res
}.toArray
}

abstract class Resolve[R: ClassTag] {
def endResolveCross(
obj: Module,
@@ -417,9 +188,12 @@ abstract class Resolve[R: ClassTag] {
endResolveLabel(obj, last, discover, rest)

case head :: tail =>
def recurse(searchModules: Seq[Module], resolveFailureMsg: => Left[String, Nothing]) = {
def recurse(
searchModules: Seq[Module],
resolveFailureMsg: => Left[String, Nothing]
): Either[String, Seq[R]] = {
val matching = searchModules
.map(resolve(tail, _, discover, rest, remainingCrossSelectors))
.map(m => resolve(tail, m, discover, rest, remainingCrossSelectors))

matching match {
case Seq(Left(err)) => Left(err)
@@ -473,20 +247,24 @@ abstract class Resolve[R: ClassTag] {
case Segment.Cross(cross) =>
obj match {
case c: Cross[Module] =>
recurse(
val searchModules =
if (cross == Seq("__")) for ((_, v) <- c.items) yield v
else if (cross.contains("_")) {
for {
(k, v) <- c.items
if k.length == cross.length
if k.zip(cross).forall { case (l, r) => l == r || r == "_" }
} yield v
} else c.itemMap.get(cross.toList).toSeq,
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
cross.map(_.toString),
obj.millModuleSegments.value
)
} else c.itemMap.get(cross.toList).toSeq

recurse(
searchModules = searchModules,
resolveFailureMsg =
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
cross.map(_.toString),
obj.millModuleSegments.value
)
)
case _ =>
Left(
80 changes: 80 additions & 0 deletions main/src/mill/main/ResolveMetadata.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package mill.main

import mill.define._

object ResolveMetadata extends Resolve[String] {
def singleModuleMeta(obj: Module, discover: Discover[_], isRootModule: Boolean): Seq[String] = {
val modules = obj.millModuleDirectChildren.map(_.toString)
val targets =
obj
.millInternal
.reflectAll[NamedTask[_]]
.map(_.toString)
val commands =
for {
(cls, entryPoints) <- discover.value
if cls.isAssignableFrom(obj.getClass)
ep <- entryPoints
} yield
if (isRootModule) ep._2.name
else s"$obj.${ep._2.name}"

modules ++ targets ++ commands
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[String]] = {
def direct = singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty)
last match {
case "__" =>
Right(
// Filter out our own module in
obj.millInternal.modules.flatMap(m => singleModuleMeta(m, discover, m == obj))
)
case "_" => Right(direct)
case _ =>
direct.find(_.split('.').last == last) match {
case None =>
Resolve.errorMsgLabel(direct, Seq(Segment.Label(last)), obj.millModuleSegments.value)
case Some(s) => Right(Seq(s))
}
}
}

def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, List[String]] = {
obj match {
case c: Cross[Module] =>
last match {
case List("__") => Right(c.items.map(_._2.toString))
case items =>
c.items
.filter(_._1.length == items.length)
.filter(_._1.zip(last).forall { case (a, b) => b == "_" || a.toString == b })
.map(_._2.toString) match {
case Nil =>
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
last,
obj.millModuleSegments.value
)
case res => Right(res)
}

}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}
}
75 changes: 75 additions & 0 deletions main/src/mill/main/ResolveSegments.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package mill.main

import mill.define._
import mill.main.ResolveMetadata.singleModuleMeta

object ResolveSegments extends Resolve[Segments] {

override def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[Segments]] = {
obj match {
case c: Cross[Module] =>
last match {
case List("__") => Right(c.items.map(_._2.millModuleSegments))
case items =>
c.items
.filter(_._1.length == items.length)
.filter(_._1.zip(last).forall { case (a, b) => b == "_" || a.toString == b })
.map(_._2.millModuleSegments) match {
case Nil =>
Resolve.errorMsgCross(
c.items.map(_._1.map(_.toString)),
last,
obj.millModuleSegments.value
)
case res => Right(res)
}
}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[Segments]] = {
val target =
obj
.millInternal
.reflectSingle[Target[_]](last)
.map(t => Right(t.ctx.segments))

val command =
Resolve
.invokeCommand(obj, last, discover.asInstanceOf[Discover[Module]], rest)
.headOption
.map(_.map(_.ctx.segments))

val module =
obj.millInternal
.reflectNestedObjects[Module]
.find(_.millOuterCtx.segment == Segment.Label(last))
.map(m => Right(m.millModuleSegments))

command orElse target orElse module match {
case None =>
Resolve.errorMsgLabel(
singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty),
Seq(Segment.Label(last)),
obj.millModuleSegments.value
)

case Some(either) => either.right.map(Seq(_))
}
}
}
82 changes: 82 additions & 0 deletions main/src/mill/main/ResolveTasks.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package mill.main

import mill.define._
import mill.main.ResolveMetadata.singleModuleMeta

object ResolveTasks extends Resolve[NamedTask[Any]] {

def endResolveCross(
obj: Module,
last: List[String],
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[NamedTask[Any]]] = {
obj match {
case _: Cross[Module] =>
Resolve.runDefault(obj, Segment.Cross(last), discover, rest).flatten.headOption match {
case None =>
Left(
"Cannot find default task to evaluate for module " +
Segments((obj.millModuleSegments.value :+ Segment.Cross(last)): _*).render
)
case Some(v) => v.map(Seq(_))
}
case _ =>
Left(
Resolve.unableToResolve(Segment.Cross(last), obj.millModuleSegments.value) +
Resolve.hintListLabel(obj.millModuleSegments.value)
)
}
}

def endResolveLabel(
obj: Module,
last: String,
discover: Discover[_],
rest: Seq[String]
): Either[String, Seq[NamedTask[Any]]] = last match {
case "__" =>
Right(
obj.millInternal.modules
.filter(_ != obj)
.flatMap(m => m.millInternal.reflectAll[NamedTask[_]])
)
case "_" => Right(obj.millInternal.reflectAll[NamedTask[_]])

case _ =>
val target =
obj
.millInternal
.reflectSingle[NamedTask[_]](last)
.map(Right(_))

val command = Resolve.invokeCommand(
obj,
last,
discover.asInstanceOf[Discover[Module]],
rest
).headOption

command
.orElse(target)
.orElse {
Resolve.runDefault(
obj,
Segment.Label(last),
discover,
rest
).flatten.headOption
} match {
case None =>
Resolve.errorMsgLabel(
singleModuleMeta(obj, discover, obj.millModuleSegments.value.isEmpty),
Seq(Segment.Label(last)),
obj.millModuleSegments.value
)

// Contents of `either` *must* be a `Task`, because we only select
// methods returning `Task` in the discovery process
case Some(either) => either.map(Seq(_))
}
}
}
21 changes: 14 additions & 7 deletions main/src/mill/modules/Jvm.scala
Original file line number Diff line number Diff line change
@@ -36,8 +36,8 @@ import scala.annotation.tailrec

object Jvm {

private val ConcurrentRetryCount = 5
private val ConcurrentRetryWait = 100
private val CoursierRetryCount = 5
private val CoursierRetryWait = 100

/**
* Runs a JVM subprocess with the given configuration and returns a
@@ -583,7 +583,7 @@ object Jvm {

@tailrec def load(
artifacts: Seq[coursier.util.Artifact],
retry: Int = ConcurrentRetryCount
retry: Int = CoursierRetryCount
): (Seq[ArtifactError], Seq[File]) = {
import scala.concurrent.ExecutionContext.Implicits.global
val loadedArtifacts = Gather[Task].gather(
@@ -597,12 +597,19 @@ object Jvm {
}
val successes = loadedArtifacts.collect { case (_, Right(x)) => x }

if (retry > 0 && errors.exists(_.describe.contains("concurrent download"))) {
if (retry > 0 && errors.exists(e => e.describe.contains("concurrent download"))) {
ctx.foreach(_.log.debug(
s"Detected a concurrent download issue in coursier. Attempting a retry (${retry} left)"
))
Thread.sleep(ConcurrentRetryWait)
Thread.sleep(CoursierRetryWait)
load(artifacts, retry - 1)
} else if (retry > 0 && errors.exists(e => e.describe.contains("checksum not found"))) {
ctx.foreach(_.log.debug(
s"Detected a checksum download issue in coursier. Attempting a retry (${retry} left)"
))
Thread.sleep(CoursierRetryWait)
load(artifacts, retry - 1)

} else (errors, successes)
}

@@ -683,7 +690,7 @@ object Jvm {
import scala.concurrent.ExecutionContext.Implicits.global

// Workaround for https://github.com/com-lihaoyi/mill/issues/1028
@tailrec def retriedResolution(count: Int = ConcurrentRetryCount): Resolution = {
@tailrec def retriedResolution(count: Int = CoursierRetryCount): Resolution = {
val resolution = start.process.run(fetch).unsafeRun()
if (
count > 0 &&
@@ -693,7 +700,7 @@ object Jvm {
ctx.foreach(_.log.debug(
s"Detected a concurrent download issue in coursier. Attempting a retry (${count} left)"
))
Thread.sleep(ConcurrentRetryWait)
Thread.sleep(CoursierRetryWait)
retriedResolution(count - 1)
} else resolution
}
16 changes: 14 additions & 2 deletions main/test/src/eval/CrossTests.scala
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
package mill.eval

import mill.define.Discover
import mill.util.TestEvaluator

import mill.util.{TestEvaluator, TestGraphs}
import mill.util.TestGraphs.{crossResolved, doubleCross, nestedCrosses, singleCross}
import utest._
object CrossTests extends TestSuite {
@@ -50,5 +49,18 @@ object CrossTests extends TestSuite {
val Right(("212_js", 1)) = check.apply(nestedCrosses.cross("212").cross2("js").suffix)
val Right(("212_native", 1)) = check.apply(nestedCrosses.cross("212").cross2("native").suffix)
}

"nestedTaskCrosses" - {
val model = TestGraphs.nestedTaskCrosses
val check = new TestEvaluator(model)

val Right(("210_jvm_1", 1)) = check.apply(model.cross1("210").cross2("jvm").suffixCmd("1"))
val Right(("210_js_2", 1)) = check.apply(model.cross1("210").cross2("js").suffixCmd("2"))
val Right(("211_jvm_3", 1)) = check.apply(model.cross1("211").cross2("jvm").suffixCmd("3"))
val Right(("211_js_4", 1)) = check.apply(model.cross1("211").cross2("js").suffixCmd("4"))
val Right(("212_jvm_5", 1)) = check.apply(model.cross1("212").cross2("jvm").suffixCmd("5"))
val Right(("212_js_6", 1)) = check.apply(model.cross1("212").cross2("js").suffixCmd("6"))
val Right(("212_native_7", 1)) = check.apply(model.cross1("212").cross2("native").suffixCmd("7"))
}
}
}
51 changes: 46 additions & 5 deletions main/test/src/main/MainTests.scala
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
package mill.main

import mill.define.{Discover, Segment, Task}
import mill.define.{Discover, NamedTask, Segment, SelectMode, Task}
import mill.util.TestGraphs._

import utest._
object MainTests extends TestSuite {

def check[T <: mill.define.BaseModule](module: T)(
selectorString: String,
expected0: Either[String, Seq[T => Task[_]]]
expected0: Either[String, Seq[T => NamedTask[_]]]
) = checkSeq(module)(Seq(selectorString), expected0)

def checkSeq[T <: mill.define.BaseModule](module: T)(
selectorStrings: Seq[String],
expected0: Either[String, Seq[T => NamedTask[_]]]
) = {

val expected = expected0.map(_.map(_(module)))
val resolved = for {
selectors <- mill.define.ParseArgs(Seq(selectorString), multiSelect = false).map(_._1.head)
selectors <- mill.define.ParseArgs(selectorStrings, SelectMode.Single).map(_.head._1.head)
crossSelectors = selectors._2.value.map {
case Segment.Cross(x) => x.toList.map(_.toString)
case _ => Nil
@@ -26,8 +30,11 @@ object MainTests extends TestSuite {
crossSelectors.toList
)
} yield task
assert(resolved == expected)
// doesn't work for commands, don't know why
// assert(resolved == expected)
assert(resolved.map(_.map(_.toString)) == expected.map(_.map(_.toString)))
}

val tests = Tests {
val graphs = new mill.util.TestGraphs()
import graphs._
@@ -283,6 +290,10 @@ object MainTests extends TestSuite {
"cross[211].cross2[jvm].suffix",
Right(Seq(_.cross("211").cross2("jvm").suffix))
)
"pos2NoDefaultTask" - check(
"cross[211].cross2[jvm]",
Left("Cannot find default task to evaluate for module cross[211].cross2[jvm]")
)
"wildcard" - {
"first" - check(
"cross[_].cross2[jvm].suffix",
@@ -316,6 +327,36 @@ object MainTests extends TestSuite {
)
}
}

"nestedCrossTaskModule" - {
val check = MainTests.checkSeq(nestedTaskCrosses) _
"pos1" - check(
Seq("cross1[210].cross2[js].suffixCmd"),
Right(Seq(_.cross1("210").cross2("js").suffixCmd()))
)
"pos1Default" - check(
Seq("cross1[210].cross2[js]"),
Right(Seq(_.cross1("210").cross2("js").suffixCmd()))
)
// does not work because we're reflecting the Module for `def` without args,
// which misses command with args :-(
// "pos1WithWildcard" - check(
// Seq("cross1[210].cross2[js]._"),
// Right(Seq(_.cross1("210").cross2("js").suffixCmd()))
// )
"pos1WithArgs" - check(
Seq("cross1[210].cross2[js].suffixCmd", "suffix-arg"),
Right(Seq(_.cross1("210").cross2("js").suffixCmd("suffix-arg")))
)
"pos2" - check(
Seq("cross1[211].cross2[jvm].suffixCmd"),
Right(Seq(_.cross1("211").cross2("jvm").suffixCmd()))
)
"pos2Default" - check(
Seq("cross1[211].cross2[jvm]"),
Right(Seq(_.cross1("211").cross2("jvm").suffixCmd()))
)
}
}
}
}
1 change: 1 addition & 0 deletions main/test/src/util/ParseArgsTest.scala
Original file line number Diff line number Diff line change
@@ -276,6 +276,7 @@ object ParseArgsTest extends TestSuite {
def parsed(args: String*) = ParseArgs(args, selectMode)
test("rejectEmpty") {
assert(parsed("") == Left("Selector cannot be empty"))
assert(parsed() == Left("Selector cannot be empty"))
}
def check(
input: Seq[String],
22 changes: 21 additions & 1 deletion main/test/src/util/TestGraphs.scala
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package mill.util
import TestUtil.test
import mill.define.{Cross, Discover}
import mill.define.{Command, Cross, Discover, TaskModule}
import mill.{Module, T}

/**
@@ -264,6 +264,26 @@ object TestGraphs {
}
}

object nestedTaskCrosses extends TestUtil.BaseModule {
// this is somehow necessary to let Discover see our inner (default) commands
// I expected, that the identical inherited `millDiscover` is enough, but it isn't
override lazy val millDiscover: Discover[this.type] = Discover[this.type]
object cross1 extends mill.Cross[Cross1]("210", "211", "212")

class Cross1(scalaVersion: String) extends mill.Module {

object cross2 extends mill.Cross[Cross2]("jvm", "js", "native")

class Cross2(platform: String) extends mill.Module with TaskModule {
override def defaultCommandName(): String = "suffixCmd"
def suffixCmd(suffix: String = "default"): Command[String] = T.command {
scalaVersion + "_" + platform + "_" + suffix
}
}

}
}

object StackableOverrides extends TestUtil.BaseModule {
trait X extends Module {
def f = T { 1 }
28 changes: 26 additions & 2 deletions readme.adoc
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@
:link-compare: https://github.com/com-lihaoyi/mill/compare
:link-pr: {link-github}/pull
:link-issue: {link-github}/issues
:example-scala-version: 3.0.2
:example-scala-version: 3.2.0

{link-github}/actions/workflows/actions.yml[image:{link-github}/actions/workflows/actions.yml/badge.svg[Build and Release]]
{link-gitter}?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge[image:https://badges.gitter.im/Join%20Chat.svg[Gitter Chat]]
@@ -232,17 +232,41 @@ corresponding version of Mill.

=== 'main' branch
:version: main
:prev-version: 0.10.8
:milestone: 68
:milestone-name: after 0.10.8

_Changes since {prev-version}:_

_For details refer to
{link-milestone}/{milestone}?closed=1[milestone {milestone-name}]
and the {link-compare}/{prev-version}\...{version}[list of commits]._


=== 0.10.8 - 2022-10-10
:version: 0.10.8
:prev-version: 0.10.7
:milestone: 67
:milestone-name: after 0.10.7
:milestone-name: 0.10.8

_Changes since {prev-version}:_

* Improvements for better Scala 3.2 support
* Fixed non-working default commands in cross modules
* `CoursierModule`: mitigate more download failure situations (e.g. checksum failures)
* `PublishModule`: properly show `gpg` output in server mode
* `BSP`: Better compiler message handling (`logMessage` instead of `showMessage`) and support for diagnostic code
* `ScoverageModule`: Support for Scoverage 2.x
* New contrib module `GitlabPublishModule`
* Various internal improvements and version bumps
* Documentation improvements

_For details refer to
{link-milestone}/{milestone}?closed=1[milestone {milestone-name}]
and the {link-compare}/{prev-version}\...{version}[list of commits]._



=== 0.10.7 - 2022-08-24
:version: 0.10.7
:prev-version: 0.10.6
4 changes: 4 additions & 0 deletions scalalib/src/PublishModule.scala
Original file line number Diff line number Diff line change
@@ -177,6 +177,8 @@ trait PublishModule extends JavaModule { outer =>
readTimeout,
connectTimeout,
T.log,
T.workspace,
T.env,
awaitTimeout,
stagingRelease
).publish(artifacts.map { case (a, b) => (a.path, b) }, artifactInfo, release)
@@ -236,6 +238,8 @@ object PublishModule extends ExternalModule {
readTimeout,
connectTimeout,
T.log,
T.workspace,
T.env,
awaitTimeout,
stagingRelease
).publishAll(
34 changes: 31 additions & 3 deletions scalalib/src/publish/SonatypePublisher.scala
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@ import java.math.BigInteger
import java.security.MessageDigest

import mill.api.Logger
import mill.modules.Jvm
import os.Shellable

class SonatypePublisher(
@@ -15,9 +16,37 @@ class SonatypePublisher(
readTimeout: Int,
connectTimeout: Int,
log: Logger,
workspace: os.Path,
env: Map[String, String],
awaitTimeout: Int,
stagingRelease: Boolean = true
stagingRelease: Boolean
) {
@deprecated("Use other constructor instead", since = "mill 0.10.8")
def this(
uri: String,
snapshotUri: String,
credentials: String,
signed: Boolean,
gpgArgs: Seq[String],
readTimeout: Int,
connectTimeout: Int,
log: Logger,
awaitTimeout: Int,
stagingRelease: Boolean = true
) = this(
uri = uri,
snapshotUri = snapshotUri,
credentials = credentials,
signed = signed,
gpgArgs = gpgArgs,
readTimeout = readTimeout,
connectTimeout = connectTimeout,
log = log,
workspace = os.pwd,
env = sys.env,
awaitTimeout = awaitTimeout,
stagingRelease = stagingRelease
)

private val api = new SonatypeHttpApi(
uri,
@@ -174,8 +203,7 @@ class SonatypePublisher(
val fileName = file.toString
val command = "gpg" +: args :+ fileName

os.proc(command.map(v => v: Shellable))
.call(stdin = os.Inherit, stdout = os.Inherit, stderr = os.Inherit)
Jvm.runSubprocess(command, env, workspace)
os.Path(fileName + ".asc")
}

48 changes: 24 additions & 24 deletions scalalib/test/src/HelloWorldTests.scala
Original file line number Diff line number Diff line change
@@ -17,19 +17,19 @@ import utest.framework.TestPath

object HelloWorldTests extends TestSuite {

val scala2106Version = "2.10.6"
val scala21111Version = "2.11.11"
val scala210Version = sys.props.getOrElse("TEST_SCALA_2_10_VERSION", ???)
val scala211Version = sys.props.getOrElse("TEST_SCALA_2_11_VERSION", ???)
val scala2123Version = "2.12.3"
val scala2126Version = "2.12.6"
val scala2131Version = "2.13.1"
val scala212Version = sys.props.getOrElse("TEST_SCALA_2_12_VERSION", ???)
val scala213Version = sys.props.getOrElse("TEST_SCALA_2_13_VERSION", ???)

trait HelloBase extends TestUtil.BaseModule {
override def millSourcePath: os.Path =
TestUtil.getSrcPathBase() / millOuterCtx.enclosing.split('.')
}

trait HelloWorldModule extends scalalib.ScalaModule {
def scalaVersion = scala2126Version
def scalaVersion = scala212Version
}

trait HelloWorldModuleWithMain extends HelloWorldModule {
@@ -41,11 +41,11 @@ object HelloWorldTests extends TestSuite {
}
object CrossHelloWorld extends HelloBase {
object core extends Cross[HelloWorldCross](
scala2106Version,
scala21111Version,
scala210Version,
scala211Version,
scala2123Version,
scala2126Version,
scala2131Version
scala212Version,
scala213Version
)
class HelloWorldCross(val crossScalaVersion: String) extends CrossScalaModule
}
@@ -207,7 +207,7 @@ object HelloWorldTests extends TestSuite {

object HelloWorldScalaOverride extends HelloBase {
object core extends HelloWorldModule {
override def scalaVersion: Target[String] = scala2131Version
override def scalaVersion: Target[String] = scala213Version
}
}

@@ -241,7 +241,7 @@ object HelloWorldTests extends TestSuite {

object HelloWorldMacros extends HelloBase {
object core extends ScalaModule {
def scalaVersion = scala2126Version
def scalaVersion = scala212Version

override def ivyDeps = Agg(
ivy"com.github.julien-truffaut::monocle-macro::1.4.0"
@@ -254,7 +254,7 @@ object HelloWorldTests extends TestSuite {

object HelloWorldFlags extends HelloBase {
object core extends ScalaModule {
def scalaVersion = scala2126Version
def scalaVersion = scala212Version

override def scalacOptions = super.scalacOptions() ++ Seq(
"-Ypartial-unification"
@@ -264,7 +264,7 @@ object HelloWorldTests extends TestSuite {

object HelloScalacheck extends HelloBase {
object foo extends ScalaModule {
def scalaVersion = scala2126Version
def scalaVersion = scala212Version
object test extends Tests {
override def ivyDeps = Agg(ivy"org.scalacheck::scalacheck:1.13.5")
override def testFramework = "org.scalacheck.ScalaCheckFramework"
@@ -340,15 +340,15 @@ object HelloWorldTests extends TestSuite {
val Right((result, evalCount)) = eval.apply(HelloWorld.core.scalaVersion)

assert(
result == scala2126Version,
result == scala212Version,
evalCount > 0
)
}
"override" - workspaceTest(HelloWorldScalaOverride) { eval =>
val Right((result, evalCount)) = eval.apply(HelloWorldScalaOverride.core.scalaVersion)

assert(
result == scala2131Version,
result == scala213Version,
evalCount > 0
)
}
@@ -509,7 +509,7 @@ object HelloWorldTests extends TestSuite {
"artifactNameCross" - {
workspaceTest(CrossHelloWorld) { eval =>
val Right((artifactName, _)) =
eval.apply(CrossHelloWorld.core(scala2131Version).artifactName)
eval.apply(CrossHelloWorld.core(scala213Version).artifactName)
assert(artifactName == "core")
}
}
@@ -545,15 +545,15 @@ object HelloWorldTests extends TestSuite {
"v210" - TestUtil.disableInJava9OrAbove("Scala 2.10 tests don't work with Java 9+")(
workspaceTest(CrossHelloWorld)(cross(
_,
scala2106Version,
s"${scala2106Version} rox"
scala210Version,
s"${scala210Version} rox"
))
)
"v211" - TestUtil.disableInJava9OrAbove("Scala 2.11 tests don't work with Java 9+")(
workspaceTest(CrossHelloWorld)(cross(
_,
scala21111Version,
s"${scala21111Version} pwns"
scala211Version,
s"${scala211Version} pwns"
))
)
"v2123" - workspaceTest(CrossHelloWorld)(cross(
@@ -563,13 +563,13 @@ object HelloWorldTests extends TestSuite {
))
"v2124" - workspaceTest(CrossHelloWorld)(cross(
_,
scala2126Version,
s"${scala2126Version} leet"
scala212Version,
s"${scala212Version} leet"
))
"v2131" - workspaceTest(CrossHelloWorld)(cross(
_,
scala2131Version,
s"${scala2131Version} idk"
scala213Version,
s"${scala213Version} idk"
))
}

12 changes: 12 additions & 0 deletions scalalib/worker/src/mill/scalalib/worker/ZincDiagnosticCode.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package mill.scalalib.worker

import mill.api.internal
import mill.api.DiagnosticCode

import scala.jdk.OptionConverters._

@internal
final case class ZincDiagnosticCode(base: xsbti.DiagnosticCode) extends DiagnosticCode {
override def code: String = base.code()
override def explanation: Option[String] = base.explanation().toScala
}
5 changes: 5 additions & 0 deletions scalalib/worker/src/mill/scalalib/worker/ZincProblem.scala
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package mill.scalalib.worker

import mill.api.{Problem, ProblemPosition, Severity, internal}
import mill.api.DiagnosticCode
import scala.jdk.OptionConverters._

@internal
class ZincProblem(base: xsbti.Problem) extends Problem {
@@ -15,4 +17,7 @@ class ZincProblem(base: xsbti.Problem) extends Problem {
override def message: String = base.message()

override def position: ProblemPosition = new ZincProblemPosition(base.position())

override def diagnosticCode: Option[DiagnosticCode] =
base.diagnosticCode().toScala.map(ZincDiagnosticCode)
}
Original file line number Diff line number Diff line change
@@ -455,6 +455,7 @@ class ZincWorkerImpl(
val newReporter = reporter match {
case None => new ManagedLoggedReporter(10, logger)
case Some(r) => new ManagedLoggedReporter(10, logger) {

override def logError(problem: xsbti.Problem): Unit = {
r.logError(new ZincProblem(problem))
super.logError(problem)
10 changes: 8 additions & 2 deletions scalanativelib/src/ScalaNativeModule.scala
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@ import mill.api.{internal, Result}
import mill.define.{Target, Task}
import mill.modules.Jvm
import mill.scalalib.api.Util.{isScala3, scalaBinaryVersion}
import mill.scalalib.{Dep, DepSyntax, Lib, SbtModule, ScalaModule, TestModule}
import mill.scalalib.{CrossVersion, Dep, DepSyntax, Lib, SbtModule, ScalaModule, TestModule}
import mill.testrunner.TestRunner
import mill.scalanativelib.api._

@@ -84,7 +84,13 @@ trait ScalaNativeModule extends ScalaModule { outer =>
}

override def scalaLibraryIvyDeps = T {
if (isScala3(scalaVersion())) Agg.empty[Dep] else super.scalaLibraryIvyDeps()
super.scalaLibraryIvyDeps().map(dep =>
dep.copy(cross = dep.cross match {
case c: CrossVersion.Constant => c.copy(platformed = false)
case c: CrossVersion.Binary => c.copy(platformed = false)
case c: CrossVersion.Full => c.copy(platformed = false)
})
)
}

/** Adds [[nativeIvyDeps]] as mandatory dependencies. */
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
package hello

object Main extends App {
// tests lazy val work in Scala 3.2+
lazy val foo = 1

println("Hello " + vmName)
def vmName = sys.props("java.vm.name")
}
4 changes: 2 additions & 2 deletions scalanativelib/test/src/HelloNativeWorldTests.scala
Original file line number Diff line number Diff line change
@@ -29,8 +29,8 @@ object HelloNativeWorldTests extends TestSuite {

object HelloNativeWorld extends TestUtil.BaseModule {
val matrix = for {
scala <- Seq("3.1.0", scala213, "2.12.13", "2.11.12")
scalaNative <- Seq(scalaNative04, "0.4.3")
scala <- Seq("3.2.0", "3.1.3", scala213, "2.12.13", "2.11.12")
scalaNative <- Seq(scalaNative04, "0.4.7")
mode <- List(ReleaseMode.Debug, ReleaseMode.ReleaseFast)
if !(ZincWorkerUtil.isScala3(scala) && scalaNative == scalaNative04)
} yield (scala, scalaNative, mode)