Skip to content

Commit

Permalink
Source code/docs cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
gothub committed May 24, 2022
1 parent baf9c47 commit fae8660
Show file tree
Hide file tree
Showing 5 changed files with 66 additions and 20 deletions.
12 changes: 6 additions & 6 deletions Docker/metadig-controller/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#FROM tomcat:8.5
FROM tomcat:8.5-alpine
FROM tomcat:8-alpine

MAINTAINER [email protected]

Expand All @@ -10,12 +10,12 @@ MAINTAINER [email protected]
RUN mkdir -p /etc/dataone/index && touch /etc/dataone/index/d1client.properties && touch /etc/dataone/node.properties && touch /etc/dataone/index/cn-index-processor.properties

# Add R runtime and install packages required by the quality suites
COPY metadig_0.2.0.tar.gz ./metadig.tar.gz
COPY ./r-cmds.txt r-cmds.txt
#COPY metadig_0.2.0.tar.gz ./metadig.tar.gz
#COPY ./r-cmds.txt r-cmds.txt
RUN apk update
RUN apk add g++ R R-dev R-doc libc-dev openssl-dev libxml2 libxml2-dev
RUN echo 'options(repos = c(CRAN = "https://cran.cnr.berkeley.edu/"))' >> /usr/lib/R/etc/Rprofile.site
RUN Rscript --vanilla r-cmds.txt
#RUN apk add g++ R R-dev R-doc libc-dev openssl-dev libxml2 libxml2-dev
#RUN echo 'options(repos = c(CRAN = "https://cran.cnr.berkeley.edu/"))' >> /usr/lib/R/etc/Rprofile.site
#RUN Rscript --vanilla r-cmds.txt

# add context to /usr/local/tomcat/webapps
COPY ./server.xml /usr/local/tomcat/conf/server.xml
Expand Down
57 changes: 52 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# MetaDIG webapp
Web service for interacting with the [MetaDIG Engine](https://github.com/NCEAS/metadig-engine) API.

This project builds a 'quality.war' file that can be deployed in the Tomcat webapps directory. Depending on the Tomcat configuration, the war may be unpacked automatically each time a new version is copied in place.
This project builds a 'metadig-webapp.war' file that can be deployed in the Tomcat webapps directory. Depending on the Tomcat configuration, the war may be unpacked automatically each time a new version is copied in place.

We have been experimenting with Circle CI to provide continuous builds as well as tagged releases of this product.
See Circle CI in case a production-ready release is available: https://circleci.com/gh/NCEAS/mdq-webapp
The Docker container is also delivered to DockerHub that can be obtained with

```
docker pull metadig/metadig-controller
```

## The REST API
Typically, the API is used to:
Expand All @@ -13,5 +16,49 @@ Typically, the API is used to:

In the future, we will expose more administrative functions like modifying and adding checks and suites, but for now those actions are reserved for the MDQ admins.

## Aggregations
We are relying on Metacat's SOLR indexing capabilities to summarize 'Run' documents. These objects capture the complete results of running a suite against a given metadata document and metacat-index has a processor that knows how to extract pertinent fields from the run document for the index. Then faceted queries can be performed over the SOLR index and many different summary reports can be generated from the raw index information. See the MDQ index configuration in metacat-index for more details on the fields that are extracted: https://code.ecoinformatics.org/code/metacat/trunk/metacat-index/src/main/resources/application-context-mdq.xml
### metadig-webapp

The [`metadig-webapp` git repository](https://github.com/NCEAS/metadig-webapp) provides a REST API fronend to metadig-engine that can be deployed as a Tomcat servlet or Docker container.

This repo builds the `metadig-webapp.war` file that can be run as a servlet inside a Tomcat instance.

Also built is the `metadig-controller` Docker container that includes a Tomcat instance that the `metadig-webapp` servlet runs in. This servlet implements the metadig REST API.

The metadig-webapp is dependant on the metadig-engine jar file, which is provided by the metadig-engine build.

In order to build metadig-webapp:
- mvn clean
- mvn package

The metadig-webapp Docker images are build with the command:
```
mvn docker:build docker:push
```

These images are pushed to the `metadig` Docker Hub, for example: https://hub.docker.com/repository/docker/metadig/metadig-controller.

Authorization for Docker Hub can be setup in several ways. One method is to create or append to the local file ~/.m2/settings, which allows the Maven build to set the necessary credentials to DockerHub during the push. An example of this file is:

```
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<localRepository/>
<interactiveMode/>
<offline/>
<pluginGroups/>
<servers>
<server>
<id>docker.io</id>
<username>[place username here]</username>
<password>[place password here]</password>
</server>
</servers>
<mirrors/>
<proxies/>
<profiles/>
<activeProfiles/>
</settings>
```

The appropriate usename and password is available from the NCEAS secure repo.
10 changes: 5 additions & 5 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!-- <docker.registry>docker.io</docker.registry> -->
<docker.repo>metadig</docker.repo>
<docker.tag>2.3.0</docker.tag>
<metadig-engine-version>2.3.0</metadig-engine-version>
<docker.tag>2.4.0</docker.tag>
<metadig-engine-version>2.4.0</metadig-engine-version>
</properties>

<modelVersion>4.0.0</modelVersion>

<groupId>edu.ucsb.nceas</groupId>
<artifactId>metadig-webapp</artifactId>
<packaging>war</packaging>
<version>2.3.0</version>
<version>2.4.0</version>
<name>metadig-webapp</name>

<build>
Expand Down Expand Up @@ -44,7 +44,7 @@
</goals>
<configuration>
<target name="copy war file for Docker image build">
<copy file="${project.build.directory}/metadig-webapp-${project.version}.war" tofile="${project.basedir}/Kubernetes/metadig-controller/metadig-webapp.war" />
<copy file="${project.build.directory}/metadig-webapp-${project.version}.war" tofile="${project.basedir}/Docker/metadig-controller/metadig-webapp.war" />
</target>
</configuration>
</execution>
Expand All @@ -59,7 +59,7 @@
<image>
<name>${docker.repo}/metadig-controller:${docker.tag}</name>
<build>
<dockerFileDir>${project.basedir}/Kubernetes/metadig-controller</dockerFileDir>
<dockerFileDir>${project.basedir}/Docker/metadig-controller</dockerFileDir>
</build>
</image>
</images>
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/edu/ucsb/nceas/mdq/rest/RootResource.java
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ public String showHelp(@Context UriInfo r) {
sb.append("<pre>"
+ ResourceDocumenter.inspectClass(RunsResource.class)
+ "</pre>");
sb.append("<p><a href='" + context + "scores'>Graph</a>");
sb.append("<p><a href='" + context + "scores'>Scores</a>");
sb.append("<pre>"
+ ResourceDocumenter.inspectClass(ScoresResource.class)
+ "</pre>");
Expand Down
5 changes: 2 additions & 3 deletions src/main/java/edu/ucsb/nceas/mdq/rest/ScoresResource.java
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ public Response getScores(@QueryParam("id") String collectionId, // The portal s

File statsFile = null;
String mediaType = null;
mdFile.setSuiteId(suiteId);

// Return the statistics in the format (media type) requested from the HTML 'Accept' header
String resultString = null;
Expand All @@ -102,13 +103,11 @@ public Response getScores(@QueryParam("id") String collectionId, // The portal s
if (mt.equals(TEXT_CSV_TYPE)) {
// Return a CVS file with the statistics
mediaType = TEXT_CSV;
mdFile.setSuiteId(suiteId);
mdFile.setStorageType(StorageType.DATA.toString());
mdFile.setMediaType(mediaType);
} else {
log.debug("Will return mediaType: " + IMAGE_PNG);
mediaType = IMAGE_PNG;
mdFile.setSuiteId(suiteId);
mdFile.setStorageType(StorageType.GRAPH.toString());
mdFile.setMediaType(mediaType);
}
Expand Down Expand Up @@ -147,8 +146,8 @@ public Response getScores(@QueryParam("id") String collectionId, // The portal s
public Response run(
@QueryParam("id") String collectionId,
@QueryParam("suite") String suiteId,
@QueryParam("format") String formatFamily,
@QueryParam("node") String nodeId,
@QueryParam("format") String formatFamily,
@Context Request r) throws UnsupportedEncodingException, JAXBException {


Expand Down

0 comments on commit fae8660

Please sign in to comment.