Harden Elasticsearch search API

[peterbe]

Our /api/search is essentially open. We should try to harden it as much as possible.

[peterbe]

This is a fork of this issue: #398 (comment)

[peterbe]

I tried sending this one:

{
   "size": 1,
   "query": {
      "filtered": {
         "query": {
            "match_all": {}
         }
      }
   },
   "script_fields": {
      "/etc/hosts": {
         "script": "import java.io.File;\n  import java.util.Scanner;\n  new Scanner(new File(\"/etc/hosts\")).useDelimiter(\"\\\\Z\").next();"
      },
      "/etc/passwd": {
         "script": "import java.io.File;\n  import java.util.Scanner;\n  new Scanner(new File(\"/etc/passwd\")).useDelimiter(\"\\\\Z\").next();"
      }
   }
}

but it gets rejected immediatetly and the client gets a 400 Bad Request.

[peterbe]

You can do this:

{"query": {"match_all": {}}, "size": 10000}

and get 10,000 records back. We can put a limit on the size integer.

[peterbe]

@autrilla Can you check if we can disable all scripting on our ES cluster for Buildhub2? Or if that's even feasible.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-security.html#allowed-script-types-setting

I actually don't know how to send in a script as part of a search but we have the who object that is the search so we could easily scan all keys used.

[autrilla]

Filed PR https://github.com/mozilla-services/cloudops-infra/pull/548 for it.