Fixes for haystack indexing #57

petrklus · 2016-12-08T16:00:28Z

No description provided.

FinalAngel · 2016-12-08T16:18:22Z

@petrklus I do not understand the issue here, could you elaborate a bit further?

petrklus · 2016-12-08T18:41:54Z

@FinalAngel I am using Aldryn Search (elasticsearch etc.) but since the introduction of new Google Maps plugin version, indexing fails:

I realise that the fix supplied is quite crude, just wanted to put the issue on your radar.

/app/addons/aldryn-django/aldryn_config.py:115: RuntimeWarning: no cache configured. Falling back to CACHE_URL=locmem://
  RuntimeWarning,
Registering auto field: djangocms_link.fields.PageSearchField
Registering new field: djangocms_link.fields.PageSearchField; With actual id: 4e3e5fc5852603ee86d76969d306349444413d24
/virtualenv/lib/python2.7/site-packages/debug_toolbar/settings.py:174: Warning: Please use an explicit setup with the debug_toolbar.middleware.DebugToolbarMiddleware after django.middleware.gzip.GZipMiddlware in MIDDLEWARE_CLASSES.
  "in MIDDLEWARE_CLASSES.", Warning)

/virtualenv/lib/python2.7/site-packages/debug_toolbar/settings.py:174: Warning: Please use an explicit setup with the debug_toolbar.middleware.DebugToolbarMiddleware after django.middleware.gzip.GZipMiddlware in MIDDLEWARE_CLASSES.
  "in MIDDLEWARE_CLASSES.", Warning)

System check identified some issues:

WARNINGS:
?: (1_8.W001) The standalone TEMPLATE_* settings were deprecated in Django 1.8 and the TEMPLATES dictionary takes precedence. You must put the values of the following settings into your default TEMPLATES dict: TEMPLATE_DIRS, ALLOWED_INCLUDE_ROOTS, TEMPLATE_CONTEXT_PROCESSORS, TEMPLATE_LOADERS.
Removing all documents from your index because you said so.
DELETE http://es:9200/local-default [status:404 request:0.068s]
DELETE http://es:9200/local-fr [status:404 request:0.004s]
DELETE http://es:9200/local-es [status:404 request:0.006s]
All documents removed.
Indexing 20 titles
GET http://es:9200/local-default/_mapping [status:404 request:0.003s]
PUT http://es:9200/local-default [status:200 request:0.544s]
PUT http://es:9200/local-default/_mapping/modelresult [status:200 request:0.222s]
[ERROR/MainProcess] Failed indexing 1 - 20 (retry 5/5): 'googlemap_template' (pid 6): 'googlemap_template'
Traceback (most recent call last):
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 84, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/virtualenv/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 168, in update
    prepped_data = index.full_prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/haystack/indexes.py", line 208, in full_prepare
    self.prepared_data = self.prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/base.py", line 42, in prepare
    self.prepared_data['text'] = self.get_search_data(obj, current_language, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 114, in get_search_data
    plugin_text_content = self.get_plugin_search_text(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 130, in get_plugin_search_text
    plugin_content_bits = get_plugin_index_data(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 81, in get_plugin_index_data
    plugin_contents = _render_plugin(instance, context)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 29, in _render_plugin
    content = plugin.render_plugin(context)
  File "/virtualenv/lib/python2.7/site-packages/cms/models/pluginmodel.py", line 278, in render_plugin
    template = plugin._get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/cms/plugin_base.py", line 159, in _get_render_template
    return self.get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/djangocms_googlemap/cms_plugins.py", line 89, in get_render_template
    return 'djangocms_googlemap/{}/marker.html'.format(context['googlemap_template'])
  File "/virtualenv/lib/python2.7/site-packages/django/template/context.py", line 71, in __getitem__
    raise KeyError(key)
KeyError: 'googlemap_template'
[ERROR/MainProcess] Error updating cms using default 
Traceback (most recent call last):
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 214, in handle
    self.update_backend(label, using)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 257, in update_backend
    commit=self.commit, max_retries=self.max_retries)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 84, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/virtualenv/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 168, in update
    prepped_data = index.full_prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/haystack/indexes.py", line 208, in full_prepare
    self.prepared_data = self.prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/base.py", line 42, in prepare
    self.prepared_data['text'] = self.get_search_data(obj, current_language, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 114, in get_search_data
    plugin_text_content = self.get_plugin_search_text(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 130, in get_plugin_search_text
    plugin_content_bits = get_plugin_index_data(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 81, in get_plugin_index_data
    plugin_contents = _render_plugin(instance, context)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 29, in _render_plugin
    content = plugin.render_plugin(context)
  File "/virtualenv/lib/python2.7/site-packages/cms/models/pluginmodel.py", line 278, in render_plugin
    template = plugin._get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/cms/plugin_base.py", line 159, in _get_render_template
    return self.get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/djangocms_googlemap/cms_plugins.py", line 89, in get_render_template
    return 'djangocms_googlemap/{}/marker.html'.format(context['googlemap_template'])
  File "/virtualenv/lib/python2.7/site-packages/django/template/context.py", line 71, in __getitem__
    raise KeyError(key)
KeyError: 'googlemap_template'
Traceback (most recent call last):
  File "manage.py", line 7, in <module>
    startup.manage(path=os.path.dirname(os.path.abspath(__file__)))
  File "/virtualenv/lib/python2.7/site-packages/aldryn_django/startup.py", line 11, in manage
    utility.execute()
  File "/virtualenv/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/virtualenv/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/virtualenv/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/rebuild_index.py", line 37, in handle
    call_command('update_index', **options)
  File "/virtualenv/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/virtualenv/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 214, in handle
    self.update_backend(label, using)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 257, in update_backend
    commit=self.commit, max_retries=self.max_retries)
  File "/virtualenv/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 84, in do_update
    backend.update(index, current_qs, commit=commit)
  File "/virtualenv/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 168, in update
    prepped_data = index.full_prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/haystack/indexes.py", line 208, in full_prepare
    self.prepared_data = self.prepare(obj)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/base.py", line 42, in prepare
    self.prepared_data['text'] = self.get_search_data(obj, current_language, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 114, in get_search_data
    plugin_text_content = self.get_plugin_search_text(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 130, in get_plugin_search_text
    plugin_content_bits = get_plugin_index_data(base_plugin, request)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 81, in get_plugin_index_data
    plugin_contents = _render_plugin(instance, context)
  File "/virtualenv/lib/python2.7/site-packages/aldryn_search/helpers.py", line 29, in _render_plugin
    content = plugin.render_plugin(context)
  File "/virtualenv/lib/python2.7/site-packages/cms/models/pluginmodel.py", line 278, in render_plugin
    template = plugin._get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/cms/plugin_base.py", line 159, in _get_render_template
    return self.get_render_template(context, instance, placeholder)
  File "/virtualenv/lib/python2.7/site-packages/djangocms_googlemap/cms_plugins.py", line 89, in get_render_template
    return 'djangocms_googlemap/{}/marker.html'.format(context['googlemap_template'])
  File "/virtualenv/lib/python2.7/site-packages/django/template/context.py", line 71, in __getitem__
    raise KeyError(key)
KeyError: 'googlemap_template'

FinalAngel · 2016-12-09T08:34:16Z

@petrklus though travis doesn't like the syntax, I'd like to run this by @czpython to have some feedback if thats okay for you. He'll be back next week

FinalAngel · 2017-04-19T07:32:55Z

@czpython that was a long week :) would you be able to have a look soon?

czpython · 2017-04-19T10:29:55Z

Hello @petrklus,
Thanks for reporting this.
The issue itself lies in Aldryn Search, context does not persist after a plugin is rendered.
So the googlemap_template is always missing from the context.

I'm closing this here and will instead open an issue on Aldryn Search (divio/aldryn-search#73)

codecov-io · 2017-04-19T10:29:56Z

Codecov Report

Merging #57 into master will decrease coverage by 2.16%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #57      +/-   ##
==========================================
- Coverage    74.5%   72.33%   -2.17%     
==========================================
  Files          11       11              
  Lines         200      206       +6     
  Branches       17       17              
==========================================
  Hits          149      149              
- Misses         50       56       +6     
  Partials        1        1

Impacted Files	Coverage Δ
djangocms_googlemap/cms_plugins.py	`72.34% <0%> (-10.59%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bfbc7cc...2f04c04. Read the comment docs.

petrklus added 2 commits December 8, 2016 16:50

Fixes broken haystack indexing

49d86cb

Fix for routes as well

2f04c04

FinalAngel assigned FinalAngel and czpython Dec 9, 2016

FinalAngel requested a review from czpython January 3, 2017 13:22

FinalAngel added needs design decision needs expert opinion labels Jan 10, 2017

czpython closed this Apr 19, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes for haystack indexing #57

Fixes for haystack indexing #57

petrklus commented Dec 8, 2016

FinalAngel commented Dec 8, 2016

petrklus commented Dec 8, 2016 •

edited

Loading

FinalAngel commented Dec 9, 2016

FinalAngel commented Apr 19, 2017

czpython commented Apr 19, 2017

codecov-io commented Apr 19, 2017

Fixes for haystack indexing #57

Fixes for haystack indexing #57

Conversation

petrklus commented Dec 8, 2016

FinalAngel commented Dec 8, 2016

petrklus commented Dec 8, 2016 • edited Loading

FinalAngel commented Dec 9, 2016

FinalAngel commented Apr 19, 2017

czpython commented Apr 19, 2017

codecov-io commented Apr 19, 2017

Codecov Report

petrklus commented Dec 8, 2016 •

edited

Loading