-
Notifications
You must be signed in to change notification settings - Fork 195
SysAdmin
https://github.com/CPAN-API/network-infrastructure
- Includes disk setup / expansion instructions
sudo -u metacpan /home/metacpan/bin/metacpan-api-carton-exec bin/metacpan release --latest http://cpan.metacpan.org/authors/id/X/XS/XSAWYERX/MetaCPAN-API-0.33.tar.gz
Or, just use this shell script
bin/metacpan release --skip --age 24 --latest ~/CPAN/authors/id/
bin/metacpan release http://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.16.0.tar.bz2
bin/metacpan release --status latest http://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.16.1.tar.bz2
Or use this shell script
The above syntax will force the status bit to "latest", which we need to do manually only for new, latest Perl releases. It also reindexes the predecessor so that we don't have two Perl releases with the status of "latest".
The following services are set up:
starman_metacpan-api
starman_metacpan-web
metacpan-watcher
rrrclient-metacpan
elasticsearch-es-01
Each of those services can be restarted by calling service $name restart (as superuser).
On all machines run..
sudo /etc/puppet/run.sh.
From within the metacpan-conf-private repository:
Specific 'keys' ( as set in the code with $c->add_surrogate_key() )
perl bin/purge_fastly.pl --service web --key HOMEPAGE --key ABOUTThe entire site ( e.g. if JS or css is updated and affects every page )
perl bin/purge_fastly.pl --all --service webNote also these caching issues.
As metacpan user:
sh /home/metacpan/bin/metacpan-api-carton-exec bin/metacpan mapping
https://github.com/CPAN-API/metacpan-credentials/blob/master/hosting/bytemark.md (access limited)
The ElasticSearch status can be queried from within the box:
$ curl localhost:9200/cpan/_status?pretty
$ curl localhost:9200/_cluster/health/cpan_v1?level=shards
[22:03:08] <mo> [20:12:26] sudo -i -u metacpan # become metacpan user
[22:03:08] <mo> [20:12:32] we don't have root on that box
...
[22:03:09] <mo> [20:26:53] $ mount /mnt/backup
[22:03:09] <mo> [20:27:22] and backups are in /home/metacpan/api.metacpan.org/var/backup
[22:03:09] <mo> [20:27:33] run bin/metacpan backup to restore
Certificates are currently minted by StartSSL using their free Class 1 level process. The Class 1 level certs are good for one year, cover a top-level domain as well as a subdomain, and only require minimal personal identity information (name, email, physical address, phone). Best of all, they're free. The only caveat is that you cannot mint a cert if an existing cert exists for the same DNS name and the existing cert's expiry is more than two weeks out. To do so would first require a 25$ revocation fee for the original cert before being able to mint a new one.
https://github.com/CPAN-API/metacpan-conf-private/tree/master/ssl/metacpan-api
- server.key - Copy of the 2048-bit RSA private key
- server.csr - Certificate Signing Request sent to StartSSL
- server.pub - Public cert provided by StartSSL
- server.crt - Combined public cert + intermediate cert + StartSSL CA root
The combined server.crt file is generated using the
/etc/puppet/private/bm-n2/ssl/chain-to-startssl
script and files in /etc/puppet/private/bm-n2/ssl/startssl-ca.
The server.key for api.metacpan.org, cpan.metacpan.org, and metacpan.org is a copy of 2014-01-09.key. Sharing a private key makes renewing certs easier. Note that currently www.metacpan.org has a different key.
The containing directory, ssl/, is a local git repository. This aids
in not losing our keys or certs, which may be hard or impossible to
replace easily if overwritten during a botched update.
If you have access.. see: https://github.com/CPAN-API/metacpan-conf-private
Only api.metacpan.org needs a public certificate, as the rest use the Fastly SSL certificates for the public, and self signed certs for Fastly to talk to.
Good for api.metacpan.org - Only used by api. Minted by trs.
Renewing the certs from StartSSL requires:
- Re-validate control over metacpan.org via their process. Email to hostmaster@ or [email protected] will both go to [email protected] where you can see it.
- Skip their private key generation step and supply a CSR directly. You can and should reuse the existing CSRs to avoid problems. If you regenerate a CSR, make sure it matches the key! Compare the output of:
openssl rsa -noout -modulus < server.key | sha1sum
openssl req -noout -modulus < new.csr | sha1sum
- Save the new public cert to
<dns-name>/server.pub - Run
chain-to-startssl <dns-name>to generate a server.crt with the appropriate certificate chain. - Install new certs and restart services by running puppet:
/etc/puppet/run.sh - Manually verify that everything worked!
- Commit your changes to the local git repo:
cd /etc/puppet/private/bm-n2/ssl
sudo git add -A
sudo git commit --author='Your Name <[email protected]>'
sudo awk '{print $1}' /var/log/nginx/metacpan-api/access.log | sort | uniq -c | sort -nr | less
There's no point in searching for IPs, as they'll all point back to Fastly.
sudo awk -F"\"" '{print $6}' /var/log/nginx/metacpan-web/access.log | sort | uniq -dc | sort -nr | head -20