Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORE-69: update Scala, sbt versions and associated plugins #1566

Merged
merged 8 commits into from
Oct 16, 2024
Merged

CORE-69: update Scala, sbt versions and associated plugins #1566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e2eb2c6
update Scala, sbt versions and associated plugins
davidangb Oct 15, 2024
a0cf79f
reverse scalacOptions -Wconf values due to breaking change in Scala 2…
davidangb Oct 15, 2024
db93b61
Implicit definition should have explicit type
davidangb Oct 15, 2024
732a9e5
tests: Implicit definition should have explicit type
davidangb Oct 15, 2024
a0c6025
Merge branch 'develop' into da_CORE-69_updateScala
davidangb Oct 15, 2024
229310e
fix assembly merging
davidangb Oct 16, 2024
300d37d
fix safe.directory for pact
davidangb Oct 16, 2024
489c628
whoops, wrong dir for safe.directory
davidangb Oct 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/verify_consumer_pacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -319,8 +319,8 @@ jobs:
-e JANITOR_CLIENT_CREDENTIAL_FILE_PATH="" \
-e JANITOR_TRACK_RESOURCE_PROJECT_ID="" \
-e JANITOR_TRACK_RESOURCE_TOPIC_ID="" \
sbtscala/scala-sbt:openjdk-17.0.2_1.7.2_2.13.10 \
sbt "set scalafmtOnCompile := false" "project pact4s" "testOnly *SamProviderSpec"
sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.10.2_2.13.15 \
bash -c "git config --global --add safe.directory /working/sam && sbt \"set scalafmtOnCompile := false\" \"project pact4s\" \"testOnly *SamProviderSpec\""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here and elsewhere, set the scala-sbt docker image to the version that uses sbt 1.10.2 and Scala 2.13.15.

The docker image upgrade must have updated its version of git, because it started running into the "dubious ownership" error; so, I also added the incantation to set the git safe.directory.


can-i-deploy: # The can-i-deploy job will run as a result of a Sam PR. It reports the pact verification statuses on all deployed environments.
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion automation/Dockerfile-tests
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM sbtscala/scala-sbt:openjdk-17.0.2_1.7.2_2.13.10
FROM sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.10.2_2.13.15

COPY src /app/src
COPY test.sh /app
Expand Down
2 changes: 1 addition & 1 deletion automation/project/Settings.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ object Settings {
val commonSettings =
commonBuildSettings ++ testSettings ++ List(
organization := "org.broadinstitute.dsde.firecloud",
scalaVersion := "2.13.10",
scalaVersion := "2.13.15",
resolvers ++= commonResolvers,
scalacOptions ++= commonCompilerSettings
)
Expand Down
2 changes: 1 addition & 1 deletion automation/project/build.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=1.6.2
sbt.version=1.10.2
2 changes: 1 addition & 1 deletion codegen_java/project/build.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=1.6.2
sbt.version=1.10.2
2 changes: 1 addition & 1 deletion codegen_java/templates/libraries/okhttp-gson/build.sbt.mustache
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ lazy val root = (project in file(".")).
Seq(organization := "org.broadinstitute.dsde.workbench",
name := "sam-client",
version := createVersion("{{artifactVersion}}"),
scalaVersion := "2.13.10",
scalaVersion := "2.13.15",
scalacOptions ++= Seq("-feature"),
javacOptions in compile ++= Seq("-Xlint:deprecation"),
publishArtifact in (Compile, packageDoc) := false,
Expand Down
4 changes: 2 additions & 2 deletions docker/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ function make_jar()
GIT_MODEL_HASH=$(git log -n 1 --pretty=format:%h)

# make jar. cache sbt dependencies.
docker run --rm --link postgres:postgres -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:openjdk-17.0.2_1.7.2_2.13.10 /working/docker/init_schema.sh /working
docker run --rm --link postgres:postgres -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.10.2_2.13.15 /working/docker/init_schema.sh /working
sleep 40
docker run --rm --link postgres:postgres -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:openjdk-17.0.2_1.7.2_2.13.10 /working/docker/install.sh /working
docker run --rm --link postgres:postgres -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.10.2_2.13.15 /working/docker/install.sh /working
EXIT_CODE=$?
set -e # Turn error detection back on for the rest of the script

Expand Down
2 changes: 1 addition & 1 deletion docker/build_jar.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ set -e
# Get the last commit hash of the model directory and set it as an environment variable
GIT_MODEL_HASH=$(git log -n 1 --pretty=format:%h)

docker run --rm -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:openjdk-17.0.2_1.7.2_2.13.10 /working/docker/clean_install.sh /working
docker run --rm -e GIT_MODEL_HASH=$GIT_MODEL_HASH -v $PWD:/working -v jar-cache:/root/.ivy -v jar-cache:/root/.ivy2 sbtscala/scala-sbt:eclipse-temurin-jammy-17.0.10_7_1.10.2_2.13.15 /working/docker/clean_install.sh /working
EXIT_CODE=$?

if [ $EXIT_CODE != 0 ]; then
Expand Down
1 change: 1 addition & 0 deletions project/Merging.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ object Merging {
case PathList("META-INF", "kotlin-stdlib-common.kotlin_module") => MergeStrategy.first
case PathList("META-INF", "okio.kotlin_module") => MergeStrategy.first
case PathList("META-INF", "versions", "9", "OSGI-INF", "MANIFEST.MF") => MergeStrategy.first
case PathList("META-INF", "license", "LICENSE.mvn-wrapper.txt") => MergeStrategy.first
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some dependencies changed, such that we now have an assembly merge conflict on META-INF/license/LICENSE.mvn-wrapper.txt; added a case for this file.

case PathList("mozilla", "public-suffix-list.txt") => MergeStrategy.first
case "module-info.class" =>
MergeStrategy.discard
Expand Down
4 changes: 2 additions & 2 deletions project/Settings.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ object Settings {
"-language:existentials", // Existential types (besides wildcard types) can be written and inferred
"-unchecked", // Enable additional warnings where generated code depends on assumptions.
"-Xcheckinit", // Wrap field accessors to throw an exception on uninitialized access.
"-Wconf:cat=deprecation:ws,any:e", // Fail the compilation if there are any warnings, except for deprecation warnings.
"-Wconf:any:e,cat=deprecation:ws", // Fail the compilation if there are any warnings, except for deprecation warnings.
Copy link
Contributor Author

@davidangb davidangb Oct 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the "Breaking Changes" section of https://github.com/scala/scala/releases/tag/v2.13.15: the order in which -Wconf is parsed has changed. This change necessary to retain functionality.

Of note: I am not sure this setting is actually what we want. A few lines above we specify:

"-deprecation", // Emit warning and location for usages of deprecated APIs.

but here we configure deprecations to be ws, which stands for "warning summary" - the summary suppresses the info about where the actual deprecations are in the codebase and just gives a summary like "12 deprecations….". BUT, to keep scope of this PR down I am not changing it here; this PR preserves existing functionality.

"-Xlint:adapted-args", // Warn if an argument list is modified to match the receiver.
"-Xlint:constant", // Evaluation of a constant arithmetic expression results in an error.
"-Xlint:delayedinit-select", // Selecting member of DelayedInit.
Expand Down Expand Up @@ -68,7 +68,7 @@ object Settings {
lazy val commonSettings =
commonBuildSettings ++ commonAssemblySettings ++ commonTestSettings ++ List(
organization := "org.broadinstitute.dsde.workbench",
scalaVersion := "2.13.10",
scalaVersion := "2.13.15",
resolvers := proxyResolvers ++: resolvers.value ++: commonResolvers,
scalacOptions ++= commonCompilerSettings,
Compile / compile := (Compile / compile).dependsOn(Compile / scalafmtAll).value,
Expand Down
2 changes: 1 addition & 1 deletion project/build.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version=1.6.2
sbt.version=1.10.2
10 changes: 5 additions & 5 deletions project/plugins.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "1.2.0")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "2.3.0")

addSbtPlugin("io.spray" % "sbt-revolver" % "0.9.1")
addSbtPlugin("io.spray" % "sbt-revolver" % "0.10.0")

addSbtPlugin("org.scoverage" % "sbt-scoverage" % "2.0.6")
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "2.2.1")

addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.5.0")
addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.5.2")

addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.10.0")
addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.13.0")

addDependencyTreePlugin
2 changes: 1 addition & 1 deletion src/main/scala/org/broadinstitute/dsde/workbench/sam/api/SamRequestContextDirectives.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ trait SamRequestContextDirectives {
def withSamRequestContext: Directive1[SamRequestContext] =
for {
clientIP <- extractClientIP
otelContext <- traceRequest
otelContext <- traceRequest()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as of Scala 2.13.11, this caused compile to break. It's a warning, but we have -Wconf … any:e set, which causes warnings to be errors.

} yield SamRequestContext(Option(otelContext), clientIP.toOption)

/** Adds the route and parameters to telemetry. The route is consistent across all requests to the same endpoint. For example, if the route is
Expand Down
16 changes: 8 additions & 8 deletions src/main/scala/org/broadinstitute/dsde/workbench/sam/audit/SamAuditModel.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,10 @@ object SamAuditModelJsonSupport {
}
}

implicit val AuditInfoFormat = jsonFormat2(AuditInfo)
implicit val AuditInfoFormat: RootJsonFormat[AuditInfo] = jsonFormat2(AuditInfo)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as of Scala 2.13.11, this caused compile to break. It's a warning, but we have -Wconf … any:e set, which causes warnings to be errors.

The error was "Implicit definition should have explicit type" and we had many instances of this issue; lots of lines of code changed below.


implicit val ResourceChangeFormat = jsonFormat1(ResourceChange)
implicit val ResourceEventTypeFormat = new RootJsonFormat[ResourceEventType] {
implicit val ResourceChangeFormat: RootJsonFormat[ResourceChange] = jsonFormat1(ResourceChange)
implicit val ResourceEventTypeFormat: RootJsonFormat[ResourceEventType] = new RootJsonFormat[ResourceEventType] {
def read(obj: JsValue): ResourceEventType = obj match {
case JsString("ResourceCreated") => ResourceCreated
case JsString("ResourceParentUpdated") => ResourceParentUpdated
Expand All @@ -66,9 +66,9 @@ object SamAuditModelJsonSupport {
def write(obj: ResourceEventType): JsValue = JsString(obj.toString)
}

implicit val ResourceEventFormat = jsonFormat3(ResourceEvent)
implicit val ResourceEventFormat: RootJsonFormat[ResourceEvent] = jsonFormat3(ResourceEvent)

implicit val WorkbenchSubjectFormat = new RootJsonFormat[WorkbenchSubject] {
implicit val WorkbenchSubjectFormat: RootJsonFormat[WorkbenchSubject] = new RootJsonFormat[WorkbenchSubject] {
val MEMBER_TYPE_FIELD = "memberType"
val USER_TYPE = "user"
val GROUP_TYPE = "group"
Expand Down Expand Up @@ -105,8 +105,8 @@ object SamAuditModelJsonSupport {
}
}

implicit val AccessChangeFormat = jsonFormat5(AccessChange)
implicit val AccessChangeEventTypeFormat = new RootJsonFormat[AccessChangeEventType] {
implicit val AccessChangeFormat: RootJsonFormat[AccessChange] = jsonFormat5(AccessChange)
implicit val AccessChangeEventTypeFormat: RootJsonFormat[AccessChangeEventType] = new RootJsonFormat[AccessChangeEventType] {
def read(obj: JsValue): AccessChangeEventType = obj match {
case JsString("AccessAdded") => AccessAdded
case JsString("AccessRemoved") => AccessRemoved
Expand All @@ -116,5 +116,5 @@ object SamAuditModelJsonSupport {
def write(obj: AccessChangeEventType): JsValue = JsString(obj.toString)
}

implicit val PolicyEventFormat = jsonFormat3(AccessChangeEvent)
implicit val PolicyEventFormat: RootJsonFormat[AccessChangeEvent] = jsonFormat3(AccessChangeEvent)
}
27 changes: 15 additions & 12 deletions src/main/scala/org/broadinstitute/dsde/workbench/sam/azure/AzureModel.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,34 @@ import org.broadinstitute.dsde.workbench.model._
import org.broadinstitute.dsde.workbench.sam.model.{FullyQualifiedResourceId, ResourceAction, ResourceId}
import spray.json.DefaultJsonProtocol._
import org.broadinstitute.dsde.workbench.sam.model.api.SamJsonSupport._
import spray.json.RootJsonFormat

object AzureJsonSupport {
implicit val tenantIdFormat = ValueObjectFormat(TenantId.apply)
implicit val tenantIdFormat: ValueObjectFormat[TenantId] = ValueObjectFormat(TenantId.apply)

implicit val subscriptionIdFormat = ValueObjectFormat(SubscriptionId.apply)
implicit val subscriptionIdFormat: ValueObjectFormat[SubscriptionId] = ValueObjectFormat(SubscriptionId.apply)

implicit val managedResourceGroupNameFormat = ValueObjectFormat(ManagedResourceGroupName.apply)
implicit val managedResourceGroupNameFormat: ValueObjectFormat[ManagedResourceGroupName] = ValueObjectFormat(ManagedResourceGroupName.apply)

implicit val getPetManagedIdentityRequestFormat = jsonFormat3(GetOrCreatePetManagedIdentityRequest.apply)
implicit val getPetManagedIdentityRequestFormat: RootJsonFormat[GetOrCreatePetManagedIdentityRequest] = jsonFormat3(
GetOrCreatePetManagedIdentityRequest.apply
)

implicit val managedIdentityObjectIdFormat = ValueObjectFormat(ManagedIdentityObjectId.apply)
implicit val managedIdentityObjectIdFormat: ValueObjectFormat[ManagedIdentityObjectId] = ValueObjectFormat(ManagedIdentityObjectId.apply)

implicit val managedIdentityDisplayNameFormat = ValueObjectFormat(ManagedIdentityDisplayName.apply)
implicit val managedIdentityDisplayNameFormat: ValueObjectFormat[ManagedIdentityDisplayName] = ValueObjectFormat(ManagedIdentityDisplayName.apply)

implicit val petManagedIdentityIdFormat = jsonFormat4(PetManagedIdentityId.apply)
implicit val petManagedIdentityIdFormat: RootJsonFormat[PetManagedIdentityId] = jsonFormat4(PetManagedIdentityId.apply)

implicit val petManagedIdentityFormat = jsonFormat3(PetManagedIdentity.apply)
implicit val petManagedIdentityFormat: RootJsonFormat[PetManagedIdentity] = jsonFormat3(PetManagedIdentity.apply)

implicit val managedResourceGroupCoordinatesFormat = jsonFormat3(ManagedResourceGroupCoordinates.apply)
implicit val managedResourceGroupCoordinatesFormat: RootJsonFormat[ManagedResourceGroupCoordinates] = jsonFormat3(ManagedResourceGroupCoordinates.apply)

implicit val billingProfileIdFormat = ValueObjectFormat(BillingProfileId.apply)
implicit val billingProfileIdFormat: ValueObjectFormat[BillingProfileId] = ValueObjectFormat(BillingProfileId.apply)

implicit val actionManagedIdentityIdFormat = jsonFormat3(ActionManagedIdentityId.apply)
implicit val actionManagedIdentityIdFormat: RootJsonFormat[ActionManagedIdentityId] = jsonFormat3(ActionManagedIdentityId.apply)

implicit val actionManagedIdentityFormat = jsonFormat4(ActionManagedIdentity.apply)
implicit val actionManagedIdentityFormat: RootJsonFormat[ActionManagedIdentity] = jsonFormat4(ActionManagedIdentity.apply)
}

final case class TenantId(value: String) extends ValueObject
Expand Down
8 changes: 4 additions & 4 deletions src/main/scala/org/broadinstitute/dsde/workbench/sam/google/GoogleModel.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ package org.broadinstitute.dsde.workbench.sam.google

import cats.effect.IO
import org.broadinstitute.dsde.workbench.model.{ErrorReport, ErrorReportSource, WorkbenchEmail, WorkbenchIdentityJsonSupport}
import spray.json.DefaultJsonProtocol
import spray.json.{DefaultJsonProtocol, RootJsonFormat}

object SamGoogleModelJsonSupport {
import DefaultJsonProtocol._
import org.broadinstitute.dsde.workbench.model.ErrorReportJsonSupport._
import WorkbenchIdentityJsonSupport._

implicit val SyncReportItemFormat = jsonFormat4(SyncReportItem.apply)
implicit val SyncedPolicyFormat = jsonFormat2(SyncedPolicy.apply)
implicit val SyncReportFormat = jsonFormat1(SyncReport.apply)
implicit val SyncReportItemFormat: RootJsonFormat[SyncReportItem] = jsonFormat4(SyncReportItem.apply)
implicit val SyncedPolicyFormat: RootJsonFormat[SyncedPolicy] = jsonFormat2(SyncedPolicy.apply)
implicit val SyncReportFormat: RootJsonFormat[SyncReport] = jsonFormat1(SyncReport.apply)

}

Expand Down
8 changes: 4 additions & 4 deletions src/main/scala/org/broadinstitute/dsde/workbench/sam/model/api/ManagedGroupModel.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import monocle.macros.Lenses
import org.broadinstitute.dsde.workbench.model.{ValueObject, ValueObjectFormat, WorkbenchEmail, WorkbenchGroupName}
import org.broadinstitute.dsde.workbench.sam.model.{FullyQualifiedResourceId, ResourceId, ResourceRoleName}
import org.broadinstitute.dsde.workbench.sam.service.ManagedGroupService.MangedGroupRoleName
import spray.json.DefaultJsonProtocol
import spray.json.{DefaultJsonProtocol, RootJsonFormat}

@Lenses final case class ManagedGroupAndRole(groupName: WorkbenchGroupName, role: MangedGroupRoleName)
@Lenses final case class ManagedGroupMembershipEntry(groupName: ResourceId, role: ResourceRoleName, groupEmail: WorkbenchEmail)
Expand All @@ -21,9 +21,9 @@ object ManagedGroupModelJsonSupport {
import org.broadinstitute.dsde.workbench.model.WorkbenchIdentityJsonSupport._
import SamJsonSupport.{FullyQualifiedResourceIdFormat, ResourceIdFormat, ResourceRoleNameFormat}

implicit val ManagedGroupMembershipEntryFormat = jsonFormat3(ManagedGroupMembershipEntry.apply)
implicit val ManagedGroupMembershipEntryFormat: RootJsonFormat[ManagedGroupMembershipEntry] = jsonFormat3(ManagedGroupMembershipEntry.apply)

implicit val ManagedGroupAccessInstructionsFormat = ValueObjectFormat(ManagedGroupAccessInstructions.apply)
implicit val ManagedGroupAccessInstructionsFormat: ValueObjectFormat[ManagedGroupAccessInstructions] = ValueObjectFormat(ManagedGroupAccessInstructions.apply)

implicit val ManagedGroupSupportSummaryFormat = jsonFormat4(ManagedGroupSupportSummary.apply)
implicit val ManagedGroupSupportSummaryFormat: RootJsonFormat[ManagedGroupSupportSummary] = jsonFormat4(ManagedGroupSupportSummary.apply)
}
Loading
Loading