Skip to content
This repository has been archived by the owner on Dec 21, 2023. It is now read-only.

Commit

Permalink
code and docs update
Browse files Browse the repository at this point in the history
  • Loading branch information
dirkroorda committed Jan 13, 2017
1 parent 85abeb6 commit 2a6ba04
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 12 deletions.
12 changes: 0 additions & 12 deletions static/robots.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ Disallow: /shebanq/hebrew/verse
Disallow: /hebrew/verse
Disallow: /shebanq/hebrew/word
Disallow: /hebrew/word
Disallow: /shebanq/hebrew/words
Disallow: /hebrew/words
Disallow: /shebanq/hebrew/chart
Disallow: /hebrew/chart
Disallow: /shebanq/hebrew/item.csv
Expand All @@ -20,18 +18,8 @@ Disallow: /shebanq/hebrew/fields.json
Disallow: /hebrew/fields.json
Disallow: /shebanq/user
Disallow: /user
Disallow: /shebanq/static/images
Disallow: /static/images
Disallow: /shebanq/static/css
Disallow: /static/css
Disallow: /shebanq/static/js
Disallow: /static/js
Disallow: /shebanq/static/fonts
Disallow: /static/fonts
Disallow: /shebanq/appadmin
Disallow: /appadmin
Disallow: /shebanq/static/docs/tools/parallel/files
Disallow: /static/docs/tools/parallel/files
Disallow: /admin


Expand Down
39 changes: 39 additions & 0 deletions static/robots2017-01-13.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
User-agent: googlebot
User-agent: google
User-agent: bing
User-agent: Baiduspider
User-agent: baiduspider
User-agent: Baiduspider+
Disallow: /shebanq/hebrew/verse
Disallow: /hebrew/verse
Disallow: /shebanq/hebrew/word
Disallow: /hebrew/word
Disallow: /shebanq/hebrew/words
Disallow: /hebrew/words
Disallow: /shebanq/hebrew/chart
Disallow: /hebrew/chart
Disallow: /shebanq/hebrew/item.csv
Disallow: /hebrew/item.csv
Disallow: /shebanq/hebrew/cnotes.json
Disallow: /hebrew/cnotes.json
Disallow: /shebanq/hebrew/fields.json
Disallow: /hebrew/fields.json
Disallow: /shebanq/user
Disallow: /user
Disallow: /shebanq/static/images
Disallow: /static/images
Disallow: /shebanq/static/css
Disallow: /static/css
Disallow: /shebanq/static/js
Disallow: /static/js
Disallow: /shebanq/static/fonts
Disallow: /static/fonts
Disallow: /shebanq/appadmin
Disallow: /appadmin
Disallow: /shebanq/static/docs/tools/parallel/files
Disallow: /static/docs/tools/parallel/files
Disallow: /admin


User-agent: *
Disallow: /

0 comments on commit 2a6ba04

Please sign in to comment.