chore/merge upstream 2024112601 #79

onimsha · 2024-11-27T03:31:57Z

prevent trying to submit to jobclient when it can't take any more work (reduces log spam) (prevent trying to submit to jobclient when it can't take any more wor… onyx-dot-app/onyx#2482)
Bugfix/larger test runner (Bugfix/larger test runner onyx-dot-app/onyx#2508)
Move code block default language creation to citation processing (Move code block default language creation to citation processing onyx-dot-app/onyx#2501)
Citations prompt for slack now includes thread history (Citations prompt for slack now includes thread history onyx-dot-app/onyx#2510)
Non-spotfix deletion of users (Non-spotfix deletion of users onyx-dot-app/onyx#2499)
Added permission syncing (Added permission syncing onyx-dot-app/onyx#2340)
Assistants panel rework (Assistants panel rework onyx-dot-app/onyx#2509)
Asana connector (community originated) (Asana connector (community originated) onyx-dot-app/onyx#2485)
Added bool to disable chat_session_id check for search_docs for api
Saml account fastapi deletion (Saml account fastapi deletion onyx-dot-app/onyx#2512)
Route name (Route name onyx-dot-app/onyx#2520)
Add start date to persona (Add start date to persona onyx-dot-app/onyx#2407)
align alembic
Access type (Access type onyx-dot-app/onyx#2523)
try adding build essential (try adding build essential onyx-dot-app/onyx#2526)
Wrote tests for the chat apis (Wrote tests for the chat apis onyx-dot-app/onyx#2525)
update default search assistant selection (update default search assistant selection onyx-dot-app/onyx#2527)
Fix duplicate users from slack / web (Fix duplicate users from slack / web onyx-dot-app/onyx#2530)
Add additional custom tooling configuration (Add additional custom tooling configuration onyx-dot-app/onyx#2426)
Updated modals for clarity (Updated modals for clarity onyx-dot-app/onyx#2529)
Begin distinguishing upsert operations for clarity (Begin distinguishing upsert operations for clarity onyx-dot-app/onyx#2535)
Add support for o1 (Add support for o1 onyx-dot-app/onyx#2538)
Support svg navigation items (Support svg navigation items onyx-dot-app/onyx#2542)
Tooling testing (Tooling testing onyx-dot-app/onyx#2533)
Fix misc tool call errors (Fix misc tool call errors onyx-dot-app/onyx#2544)
test self hosted runner (test self hosted runner onyx-dot-app/onyx#2541)
Feature/runs on 2 (Feature/runs on 2 onyx-dot-app/onyx#2547)
rename classes and ignore deprecation warnings we mostly don't have c… (rename classes and ignore deprecation warnings we mostly don't have c… onyx-dot-app/onyx#2546)
fix saml email login upsert issue
Added doc_set__user_group cleanup for user_group deletion (Added doc_set__user_group cleanup for user_group deletion onyx-dot-app/onyx#2551)
Update theming (Update theming onyx-dot-app/onyx#2552)
ensure default_assistant passed through
Update merge queue logic (Update merge queue logic onyx-dot-app/onyx#2554)
Allow = in tag filter (Allow = in tag filter onyx-dot-app/onyx#2548)
fix typo (fix typo onyx-dot-app/onyx#2543)
Update some ux edge cases (Update some ux edge cases onyx-dot-app/onyx#2545)
flex none (Flex none for assistant display onyx-dot-app/onyx#2558)
ssl_ca_certs should default to None, not "". (ssl_ca_certs should default to None, not "". onyx-dot-app/onyx#2560)
update folder list (Update folder list onyx-dot-app/onyx#2563)
Feature/test pruning (Feature/test pruning onyx-dot-app/onyx#2556)
Cleaned up foreign key cleanup for user group deletion (Cleaned up foreign key cleanup for user group deletion onyx-dot-app/onyx#2559)
push to docker latest when git tag contains "latest", and tag nightly (push to docker latest when git tag contains "latest", and tag nightly onyx-dot-app/onyx#2564)
Add open in new tab for custom links (Add open in new tab for custom links onyx-dot-app/onyx#2568)
clamp retry to max_delay (clamp retry to max_delay onyx-dot-app/onyx#2570)
Added confluence permission syncing (Added confluence permission syncing onyx-dot-app/onyx#2537)
Feature/celery multi (Feature/celery multi onyx-dot-app/onyx#2470)
try user deploy key approach to tagging (try user deploy key approach to tagging onyx-dot-app/onyx#2575)
Feature/xenforo (Feature/xenforo onyx-dot-app/onyx#2497)
Minor role display refactor (Minor role display refactor onyx-dot-app/onyx#2578)
Add size limit to jira tickets (Add size limit to jira tickets onyx-dot-app/onyx#2586)
Global Curator Fix + Testing (Global Curator Fix + Testing onyx-dot-app/onyx#2591)
don't write a nightly tag to the same commit more than once (don't write a nightly tag to the same commit more than once onyx-dot-app/onyx#2585)
Minor Query History Fix (Minor Query History Fix onyx-dot-app/onyx#2594)
Show transition card if no connectors (Show transition card if no connectors onyx-dot-app/onyx#2597)
EKS initial deployment (EKS initial deployment onyx-dot-app/onyx#2154)
Fix default assistant (Fix default assistant onyx-dot-app/onyx#2600)
prevent nextFormStep unless credential fully set up (Prevent nextFormStep unless credential fully set up onyx-dot-app/onyx#2599)
Rely on PVC (Rely on PVC onyx-dot-app/onyx#2604)
Add heartbeat to indexing (Add heartbeat to indexing onyx-dot-app/onyx#2595)
Added permissions syncing for slack (Added permissions syncing for slack onyx-dot-app/onyx#2602)
Fix overflow of prompt library table (Fix overflow of prompt library table onyx-dot-app/onyx#2606)
don't push integration testing docker images (don't push integration testing docker images onyx-dot-app/onyx#2584)
Unstructured UI (Unstructured UI onyx-dot-app/onyx#2636)
Update CONTRIBUTING.md
update nav link name (update nav link name onyx-dot-app/onyx#2643)
run the nightly tag overnight relative to pacific time (run the nightly tag overnight relative to pacific time onyx-dot-app/onyx#2637)
Added quotes to project name to handle reserved words (Added quotes to project name to handle reserved words onyx-dot-app/onyx#2639)
welcome flow
raise redis connections and using blocking connection pool for more d… (raise redis connections and using blocking connection pool for more d… onyx-dot-app/onyx#2635)
add default model + minor fixes (add default model + minor fixes onyx-dot-app/onyx#2638)
sync up when checks run with branch protection required checks (sync up when checks run with branch protection required checks onyx-dot-app/onyx#2628)
Redis Cache for KV Store (Redis Cache for KV Store onyx-dot-app/onyx#2603)
Update litellm to fix bedrock models (Update litellm to fix bedrock models onyx-dot-app/onyx#2649)
fix typo
Dupe welcome modal logic (Dupe welcome modal logic onyx-dot-app/onyx#2656)
Clean, memoized assistant ordering (Clean, memoized assistant ordering onyx-dot-app/onyx#2655)
replace trivy download endpoint due to db download flakiness on their en… (replace trivy download endpoint due to db download flakiness on their en… onyx-dot-app/onyx#2661)
Better virtualization (Better virtualization onyx-dot-app/onyx#2653)
add clarity around assistants and names (Add clarity around assistants and names onyx-dot-app/onyx#2663)
[tiny] adjust user group sync log ([tiny] adjust user group sync log onyx-dot-app/onyx#2664)
Implement source testing framework + Slack (Implement source testing framework + Slack onyx-dot-app/onyx#2650)
Unstructured Update Copy (Unstructured Update Copy onyx-dot-app/onyx#2668)
delete directly via selection instead of making multiple calls to get chunk ids and delete each one (delete directly via selection instead of making multiple calls to get… onyx-dot-app/onyx#2666)
disable another flaky assert (disable another flaky assert onyx-dot-app/onyx#2678)
Allow config of background concurrency (Allow config of background concurrency onyx-dot-app/onyx#2648)
bump celery to 5.5.0b4 (bump celery to 5.5.0b4 onyx-dot-app/onyx#2681)
update_single should be optimized for a single call now (update_single should be optimized for a single call now onyx-dot-app/onyx#2671)
harden connections to redis (harden connections to redis onyx-dot-app/onyx#2677)
Improved api key forms + fix non-submittable azure (Improved api key forms + fix non-submittable azure onyx-dot-app/onyx#2654)
Add only multi tenant dependency injection (Add only multi tenant dependency injection onyx-dot-app/onyx#2588)
add multi tenant alembic (Multi tenant alembic onyx-dot-app/onyx#2589)
cors update (docker deployment cors update onyx-dot-app/onyx#2686)
Add option to adjust pool size (Add option to adjust pool size onyx-dot-app/onyx#2695)
Tenant provisioning in the dataplane (Tenant provisioning in the dataplane onyx-dot-app/onyx#2694)
disabled llm when skip_gen_ai_answer_question set (disabled llm when skip_gen_ai_answer_question set onyx-dot-app/onyx#2687)
Ensure tests don't use LLM (Ensure tests don't use LLM onyx-dot-app/onyx#2702)
Enable removal of reranking + navigate back to search settings (Enable removal of reranking + navigate back to search settings onyx-dot-app/onyx#2674)
remove left-over memo (remove left-over memo onyx-dot-app/onyx#2669)
update disabling logic ((minor) Update disabling logic onyx-dot-app/onyx#2592)
DanswerBot Update (DanswerBot Update onyx-dot-app/onyx#2697)
UX Cleanup (UX Cleanup onyx-dot-app/onyx#2701)
Feature/background prune 2 (Feature/background prune 2 onyx-dot-app/onyx#2583)
back button + popups (back button + popups onyx-dot-app/onyx#2707)
Linting (Linting onyx-dot-app/onyx#2704)
Minor update to initial modal (Minor update to initial modal onyx-dot-app/onyx#2571)
Remove Perm Sync Script Dev (Remove Perm Sync Script Dev onyx-dot-app/onyx#2712)
rely on stdout redirection for supervisord logging (rely on stdout redirection for supervisord logging onyx-dot-app/onyx#2711)
check before using fetch_versioned_implementation because it logs warnings that confuse users. (check before using fetch_versioned_implementation because it logs war… onyx-dot-app/onyx#2708)
Inline code + effect clarity (Inline code + effect clarity onyx-dot-app/onyx#2715)
add one more retry and wait a little longer to allow ourselves to recover from infra issues (add one more retry and wait a little longer to allow ourselves to rec… onyx-dot-app/onyx#2714)
better logging for actions being taken inside document_by_cc_pair_cleanup (better logging for actions being taken inside document_by_cc_pair_cle… onyx-dot-app/onyx#2713)
Fix all LegacyKeyValueFormat docker warnings
ensure all timeout -> hook (ensure all timeout -> hook onyx-dot-app/onyx#2718)
k ((minor) catch leftover dependency fix onyx-dot-app/onyx#2721)
Temp patch to remove multiple tool calls (Temp patch to remove multiple tool calls onyx-dot-app/onyx#2720)
experimental workflow to auto merge hotfixes to release branches.
Better support for image generation capable models (Better support for image generation capable models onyx-dot-app/onyx#2725)
try porting docker web build to runs-on
disable flaky test
urlencode the password part properly before putting it in the broker url (urlencode redis passwords when creating url's for celery onyx-dot-app/onyx#2719)
Handle bug with initial connector page display (Handle bug with initial connector page display onyx-dot-app/onyx#2727)
remove more flaky tests
experimental workflow to auto merge hotfixes to release branches. (experimental workflow to auto merge hotfixes to release branches. onyx-dot-app/onyx#2723)
Fix azure (Fix azure onyx-dot-app/onyx#2665)
More robust edge detection (More robust edge detection onyx-dot-app/onyx#2710)
Remove mildly annoying groups fetch (Remove mildly annoying groups fetch onyx-dot-app/onyx#2733)
add vespa + embedding timeout env variables (add vespa + embedding timeout env variables onyx-dot-app/onyx#2689)
update hotfix to use commas
rename
another cut
handle merge commits during cherry picking
need git user
add deploy key
gh cli needs its token
edit step name
fix where GH_TOKEN is set
try capturing gh output and parsing
try to update token permissions
fix pr merge command
add branching
Add sync status (Add sync status onyx-dot-app/onyx#2743)
print various celery queue lengths (print various celery queue lengths onyx-dot-app/onyx#2729)
google drive permission sync cleanup (google drive permission sync cleanup onyx-dot-app/onyx#2749)
Convert images to jpeg (Convert images to jpeg onyx-dot-app/onyx#2737)
update indexing and slack bot to use stdout options (update indexing and slack bot to use stdout options onyx-dot-app/onyx#2752)
ensure shared assistants accessible via query params (Ensure shared assistants accessible via query params onyx-dot-app/onyx#2740)
Fix slack bot follow up questions (Fix slack responses onyx-dot-app/onyx#2756)
Add tenant context (Add tenant context onyx-dot-app/onyx#2596)
PDF Encrypted Case (PDF Encrypted Case onyx-dot-app/onyx#2764)
Typo Fix (Typo Fix onyx-dot-app/onyx#2766)
Improved logging and added comments (Improved logging and added comments onyx-dot-app/onyx#2763)
Add cql support for confluence connector (Add cql support for confluence connector onyx-dot-app/onyx#2679)
account for no visible assistants (account for no visible assistants onyx-dot-app/onyx#2765)
Fix indexing model server port for warmup (Fix indexing model server port for warmup onyx-dot-app/onyx#2767)
fix typo (fix typo onyx-dot-app/onyx#2768)
fix web connector (Fix web connector onyx-dot-app/onyx#2769)
k (Indexing back button functional onyx-dot-app/onyx#2771)
k (Ensure scroll works with pasting onyx-dot-app/onyx#2772)
update several github actions to silence github deprecation warnings (update several github actions to silence github deprecation warnings onyx-dot-app/onyx#2730)
Multi tenant vespa (Multi tenant vespa onyx-dot-app/onyx#2762)
slightly cleaner loading (slightly cleaner loading onyx-dot-app/onyx#2776)
Fix tag background
Fix parallel tool calls (Fix parallel tool calls onyx-dot-app/onyx#2779)
Mypy (Mypy onyx-dot-app/onyx#2785)
Update README.md
nit (Ensure we still log indexing url onyx-dot-app/onyx#2787)
Add cursor to cql confluence (Add cursor to cql confluence onyx-dot-app/onyx#2775)
Notion Connector Improvements (Notion Connector Improvements onyx-dot-app/onyx#2789)
Fix stopping of indexing runs when pausing a connector
Handle 'cannotExportFile' + fix forms
nit (Remove dev update onyx-dot-app/onyx#2795)
Confluence permission sync fix for server deployment (Confluence permission sync fix for server deployment onyx-dot-app/onyx#2784)
Fix file too large error (Fix file too large error onyx-dot-app/onyx#2799)
Bugfix/mediawiki (Bugfix/mediawiki onyx-dot-app/onyx#2800)
Ensure warmup occurs once (Ensure warmup occurs once onyx-dot-app/onyx#2777)
Rate limiting confluence through redis (Rate limiting confluence through redis onyx-dot-app/onyx#2798)
fix index attempt refreshing automatically (fix index attempt refreshing automatically onyx-dot-app/onyx#2791)
Ensure regenerate has dropdown too (Ensure regenerate has dropdown too onyx-dot-app/onyx#2797)
ensure shared chats are shared (ensure shared chats are shared onyx-dot-app/onyx#2801)
add ca option (add ca option onyx-dot-app/onyx#2774)
various multi tenant improvements (various multi tenant improvements onyx-dot-app/onyx#2803)
improved code block copying (improved code block copying onyx-dot-app/onyx#2802)
Simpler azure embedding (Simpler azure embedding onyx-dot-app/onyx#2751)
Notion Empty Property Fix (Notion Empty Property Fix onyx-dot-app/onyx#2817)
Add custom tool headers (Add custom tool headers onyx-dot-app/onyx#2773)
Notion Recurse Empty Final Field (Notion Recurse Empty Final Field onyx-dot-app/onyx#2819)
add llama 3.2 (add llama 3.2 onyx-dot-app/onyx#2812)
Parsing (Parsing onyx-dot-app/onyx#2734)
Redirect with query param (Redirect with query param onyx-dot-app/onyx#2811)
check last_pruned instead of is_pruning (check last_pruned instead of is_pruning onyx-dot-app/onyx#2748)
ensure assistant response parsed correctly (Ensure assistant response parsed correctly onyx-dot-app/onyx#2823)
Session id: int -> UUID (Session id: int -> UUID onyx-dot-app/onyx#2814)
Add support for passthrough auth for custom tool calls (Add support for passthrough auth for custom tool calls onyx-dot-app/onyx#2824)
Added logging for when a member has no email or username
rearrange logging
prevent alembic from configuring logger (prevent alembic from configuring logger onyx-dot-app/onyx#2826)
bump litellm (bump litellm onyx-dot-app/onyx#2827)
Make flakey test still run but not fail CI
simplified the document search function
py
Bugfix/monitor exceptions (Bugfix/monitor exceptions onyx-dot-app/onyx#2830)
fix sync to use update_single (fix sync to use update_single onyx-dot-app/onyx#2822)
lock to avoid rare serializable errors (lock to avoid rare serializable errors onyx-dot-app/onyx#2818)
dont fail flaky tests
add slack markdown formatting (add slack markdown formatting onyx-dot-app/onyx#2829)
Push multi tenancy for slackbot (Push multi tenancy for slackbot onyx-dot-app/onyx#2828)
always finalize the serialized transaction so that it doesn't leak ou… (always finalize the serialized transaction so that it doesn't leak ou… onyx-dot-app/onyx#2843)
add sentry (add sentry onyx-dot-app/onyx#2786)
multitenant setup (multitenant setup onyx-dot-app/onyx#2845)
Add all-tenants migration for K8 job (Add all-tenants migration for K8 job onyx-dot-app/onyx#2846)
no serializable, use with_for_update to lock the row.
Revert "no serializable, use with_for_update to lock the row."
use native rate limiting in the confluence client (use native rate limiting in the confluence client onyx-dot-app/onyx#2837)
Refactor + add global timeout env variable (Refactor + add global timeout env variable onyx-dot-app/onyx#2844)
Cleanup connector form (Cleanup connector form onyx-dot-app/onyx#2849)
use with for update instead of serializable (use with for update instead of serializable onyx-dot-app/onyx#2848)
fresh indexing feature branch (fresh indexing feature branch onyx-dot-app/onyx#2790)
Add assistant notifications + update assistant context (Add assistant notifications + update assistant context onyx-dot-app/onyx#2816)
Empty embedding fix (Empty embedding fix onyx-dot-app/onyx#2853)
Add errors to search (Add errors to search onyx-dot-app/onyx#2854)
modify default (Update default tag version onyx-dot-app/onyx#2856)
(minor) search memoization + context ((minor) search memoization + context onyx-dot-app/onyx#2732)
update google sites + formik (update google sites + formik onyx-dot-app/onyx#2834)
missing tenant_id as optional param (missing tenant_id as optional param onyx-dot-app/onyx#2851)
push vespa managed service configs (push vespa managed service configs onyx-dot-app/onyx#2857)
ensure all sharp-related packages installed (Minor- standalone output build onyx-dot-app/onyx#2855)
Docstring Update for Docs (Docstring Update for Docs onyx-dot-app/onyx#2863)
(minor) quote overflow ((minor) quote overflow onyx-dot-app/onyx#2862)
Minor tenant ID improvements (Minor tenant ID improvements onyx-dot-app/onyx#2850)
Minor vespa standardization (Minor vespa standardization onyx-dot-app/onyx#2861)
modal onboarding clarity (Modal onboarding clarity onyx-dot-app/onyx#2780)
Make it so you can update model providers (Make it so you can update model providers onyx-dot-app/onyx#2866)
Refactored Confluence Connector (Refactored Confluence Connector onyx-dot-app/onyx#2859)
tighter signaling to prevent indexing cleanup from hitting tasks that are just starting (tighter signaling to prevent indexing cleanup from hitting tasks that are just starting onyx-dot-app/onyx#2867)
Handle models like openai/bedrock/claude-3.5-... (Handle models like openai/bedrock/claude-3.5-... onyx-dot-app/onyx#2869)
Update reset_indexes.py
add stale issues and pr's cron
Salesforce Connector Note (Salesforce Connector Note onyx-dot-app/onyx#2872)
Confluence polish (Confluence polish onyx-dot-app/onyx#2874)
Notion Fix Nested Properties (Notion Fix Nested Properties onyx-dot-app/onyx#2877)
Feature/celery refactor (Feature/celery refactor onyx-dot-app/onyx#2813)
Auth on main (Auth on main onyx-dot-app/onyx#2878)
Implement freshdesk frontend
add copying for unrecognized languages (add copying for unrecognized languages onyx-dot-app/onyx#2883)
Fix Freshdesk connector date parsing for UTC timestamps
Tiny confluence fix (Tiny confluence fix onyx-dot-app/onyx#2885)
sticky credential description (sticky credential description onyx-dot-app/onyx#2886)
Gating Notifications (Gating Notifications onyx-dot-app/onyx#2868)
Bugfix/indexing UI (Bugfix/indexing UI onyx-dot-app/onyx#2879)
add default schema config (add default schema config onyx-dot-app/onyx#2888)
Temporary fix for empty Google App credentials (Temporary fix for empty Google App credentials onyx-dot-app/onyx#2892)
Multitenant redis update (Multitenant redis update onyx-dot-app/onyx#2889)
Remove unnecessary log & Add LoadConnector
Replace html processing library with danswer util
Make ticket fetching a seperate function from processing
Remove 2 month time check & Add time range to fetch and process
Clearer azure models (Clearer azure models onyx-dot-app/onyx#2898)
working chat feedback dump script (with api addition) (working chat feedback dump script (with api addition) onyx-dot-app/onyx#2891)
update stale workflow
fix typo
More intuitive search settings interfaces (More intuitive search settings interfaces onyx-dot-app/onyx#2899)
Add global assistants context (Add global assistants context onyx-dot-app/onyx#2900)
can't add to primary_worker_locks if it doesn't exist (can't add to primary_worker_locks if it doesn't exist onyx-dot-app/onyx#2903)
try hiding celery task spam (try hiding celery task spam onyx-dot-app/onyx#2905)
Seeding (Seeding onyx-dot-app/onyx#2902)
entry["id"] can apparently be a number, so convert to str
Silence unnecessary debug log (Silence unnecessary debug log onyx-dot-app/onyx#2908)
Auto Backport Partial (Auto Backport Partial onyx-dot-app/onyx#2910)
Harmless Backport (Harmless Backport onyx-dot-app/onyx#2911)
Bugfix/connector deletion lockout (Bugfix/connector deletion lockout onyx-dot-app/onyx#2901)
Dev Experience (Dev Experience onyx-dot-app/onyx#2912)
Harmless Backport (Harmless Backport onyx-dot-app/onyx#2914)
Workflow (Workflow onyx-dot-app/onyx#2915)
Harmless Backport (Harmless Backport onyx-dot-app/onyx#2916)
Add strict json mode (Add strict json mode onyx-dot-app/onyx#2917)
Fix IT fixture ordering
Ensure proper modal fallback (Ensure proper modal fallback onyx-dot-app/onyx#2906)
Add pagination & Remove req.obj from connectors.tsx
Fix structured outputs (Fix structured outputs onyx-dot-app/onyx#2923)
Tenant integration tests (Tenant integration tests onyx-dot-app/onyx#2913)
upgraded claude model strings (upgraded claude model strings onyx-dot-app/onyx#2876)
check for index swap (check for index swap onyx-dot-app/onyx#2922)
Fixed polling issue with timezone
add posthog + layout rework (add posthog + layout rework onyx-dot-app/onyx#2926)
Backport Test (Backport onyx-dot-app/onyx#2930)
Backport Branch Fix (Backport Branch Fix onyx-dot-app/onyx#2931)
feat: sentry updates (feat: sentry updates onyx-dot-app/onyx#2929)
Backport Test (Backport Test onyx-dot-app/onyx#2935)
Backport Auth (Backport Auth onyx-dot-app/onyx#2936)
ensure proper sentry silencing (ensure proper sentry silencing onyx-dot-app/onyx#2934)
Backport Permissions (Backport Permissions onyx-dot-app/onyx#2937)
Harmless Backport Test (Harmless Backport Test onyx-dot-app/onyx#2938)
Backport Test (Backport Test onyx-dot-app/onyx#2939)
Backport Test (Backport Test onyx-dot-app/onyx#2940)
Backport Correct Branch (Backport Correct Branch onyx-dot-app/onyx#2941)
Backport Test Final (Backport Test Final onyx-dot-app/onyx#2942)
remove rt + home-grown sitemap parsing (remove rt + home-grown sitemap parsing onyx-dot-app/onyx#2933)
Tenant aware JWT strategy (Tenant aware JWT strategy onyx-dot-app/onyx#2943)
Move user fetching to SS + parallelize some server-side calls (Move user fetching to SS + parallelize some server-side calls onyx-dot-app/onyx#2932)
Clean up code
Feat: add clean logging for api routes (Feat: add clean logging for api routes onyx-dot-app/onyx#2928)
Refactor & cleanup code, process tickets in batches
a (Add cloning of message state onyx-dot-app/onyx#2815)
Onboarding nits (Onboarding nits onyx-dot-app/onyx#2907)
tenant seeding docs (tenant seeding docs onyx-dot-app/onyx#2925)
avoid image generation tool confusion
nit
Harmless Backport Test (Backport Test onyx-dot-app/onyx#2950)
Ensure build args passed to cloud web images (Ensure build args passed to cloud web images onyx-dot-app/onyx#2947)
Trigger from Workflow (Trigger from Workflow onyx-dot-app/onyx#2951)
Test Workflow Trigger (Test Workflow Trigger onyx-dot-app/onyx#2952)
Update docker-build-push-backend-container-on-tag.yml
Backport Tag Test (Backport Tag Test onyx-dot-app/onyx#2954)
GHA Trigger (GHA Trigger onyx-dot-app/onyx#2955)
Backport (Backport onyx-dot-app/onyx#2956)
Backport Test (Backport Test onyx-dot-app/onyx#2957)
Backport Test
Backport Test (Backport Test onyx-dot-app/onyx#2958)
Backport No Tag
Notion Child Block Fix (Notion Child Block Fix onyx-dot-app/onyx#2953)
Backport Richard
Backport Test (Backport Test onyx-dot-app/onyx#2959)
Add back Backport Tags
Backport Touchup
Touchup (Backport onyx-dot-app/onyx#2961)
Echo Merge Commit (Echo Merge Commit onyx-dot-app/onyx#2962)
Backport Test (Backport Test onyx-dot-app/onyx#2963)
Backport Debugging
Backport Test Conn (Backport Test onyx-dot-app/onyx#2964)
New Credentials GH
Backport Test (Backport Test onyx-dot-app/onyx#2966)
Revert
Backport Test (Backport Test onyx-dot-app/onyx#2967)
Add Conditional
Backport Test 4 (Backport Test 4 onyx-dot-app/onyx#2968)
k
Push Tag
Backport Test 5 (Backport Test 5 onyx-dot-app/onyx#2969)
Richard Key
Backport Test 6 (Backport Test 6 onyx-dot-app/onyx#2970)
Backport Final
Backport Test 7 (Backport Test 7 onyx-dot-app/onyx#2971)
Clean up & comment fetch_tickets
Bugfix/celery light backoff (Bugfix/celery light backoff onyx-dot-app/onyx#2880)
Fix document time parsing
Fix per_page count
change test port to 8889 (docker desktop is now using port 8888 which blocks the test from working on mac) (change test port to 8889 (docker desktop is now using port 8888 which… onyx-dot-app/onyx#2972)
Fix nagging double auth issue (Fix nagging double auth issue onyx-dot-app/onyx#2960)
Distinguish users in posthog (Distinguish users in posthog onyx-dot-app/onyx#2965)
Cleaner initial chat screen (Cleaner initial chat screen onyx-dot-app/onyx#2528)
add srem and sadd to tenant wrapper (add srem and sadd to tenant wrapper onyx-dot-app/onyx#2973)
add indent to scan_iter (Add indent to scan_iter onyx-dot-app/onyx#2948)
remove empty directory (remove empty directory onyx-dot-app/onyx#2977)
missing scard
fail gracefully on provider fetch (fail gracefully on provider fetch onyx-dot-app/onyx#2981)
add additional configuration options (add additional configuration options onyx-dot-app/onyx#2980)
fixed label filter (fixed label filter onyx-dot-app/onyx#2978)
Tiny logging clarity improvement (Tiny logging clarity improvement onyx-dot-app/onyx#2985)
Temp fix to add retries to get_all_vespa_ids_for_document_id
init sqlalchemy in child process (init sqlalchemy in child process onyx-dot-app/onyx#2987)
add extra tags to pruning logs (add extra tags to pruning logs onyx-dot-app/onyx#2994)
add regeneration clarity (add regeneration clarity onyx-dot-app/onyx#2986)
Add alternative auth header (Add alternative auth header onyx-dot-app/onyx#2999)
Refactor metadata
Fix typing errors
clarity (Pruning + Refresh Clarity onyx-dot-app/onyx#3001)
Minor theming (Minor theming onyx-dot-app/onyx#2993)
Disable Mediawiki Tests (Disable Mediawiki Tests onyx-dot-app/onyx#3005)
only warmup on index swap (only warmup on index swap onyx-dot-app/onyx#3003)
Gmail Connector Robustify (Gmail Connector Robustify onyx-dot-app/onyx#3000)
Feature/indexing logs (Feature/indexing logs onyx-dot-app/onyx#3002)
Update connector.py
Fix model test action name
Update README.md (chore: Added back to top , contributors section and star history graph onyx-dot-app/onyx#3011)
remove endpoint (Remove unused endpoint onyx-dot-app/onyx#3014)
Proper tenant reset (Proper tenant reset onyx-dot-app/onyx#3015)
Remove License Issues (Remove License Issues onyx-dot-app/onyx#3013)
Refactored Google Drive Connector + Permission Syncing (Refactored Google Drive Connector + Permission Syncing onyx-dot-app/onyx#2945)
Fix pywikibot droppings (Fix pywikibot droppings onyx-dot-app/onyx#2924)
Quicky
Do not count API keys as users (Do not count API keys as users onyx-dot-app/onyx#3022)
Basic multi tenant api key (Basic multi tenant api key onyx-dot-app/onyx#3004)
Feat/certificate (Feat/certificate onyx-dot-app/onyx#2998)
Fix answer with specified doc ids (Fix answer with specified doc ids onyx-dot-app/onyx#2703)
minor foreign key update (minor foreign key update onyx-dot-app/onyx#3007)
Tremor -> Shadcn (Tremor -> Shadcn onyx-dot-app/onyx#2983)
experimental github action to scan licenses
we didn't checkout the code, no trivy ignore
don't scan the os packages
Update connector.py
try repo level scan
try installing npm deps
codel permissions?
manual only for now
try pip-license-checker
fix run key
mypy fixes
more mypy fixes
out of scope
out of scope
Cleaned up connector
renamed variables
made id more unique
ID fix
troll logger statement
Super user (Super user onyx-dot-app/onyx#2944)
Feature/redis connector refactor (Feature/redis connector refactor onyx-dot-app/onyx#2992)
minor nit
Upgrade to latest NextJS + switch to turbopack (Upgrade to latest NextJS + switch to turbopack onyx-dot-app/onyx#3027)
temporary update (Include File 61 temporarily onyx-dot-app/onyx#3037)
Revert temporary modifications (Revert temporary modifications onyx-dot-app/onyx#3038)
add validated + reformatted dynamic beat acquisition (add validated + reformatted dynamic beat acquisition onyx-dot-app/onyx#3006)
slight upgrade to image generation prompts (slight upgrade to image generation prompts onyx-dot-app/onyx#3036)
add CSV display (add CSV display onyx-dot-app/onyx#3028)
Tool call per message (Tool call per message onyx-dot-app/onyx#3025)
Last Message Too Large Logging (Last Message Too Large Logging onyx-dot-app/onyx#3039)
add POSTGRES_IDLE_SESSIONS_TIMEOUT (add POSTGRES_IDLE_SESSIONS_TIMEOUT onyx-dot-app/onyx#3019)
Add drive sections (Add drive sections onyx-dot-app/onyx#3040)
add multiple formats to tools (add multiple formats to tools onyx-dot-app/onyx#3041)
Misc color clean up (Misc color clean up onyx-dot-app/onyx#3026)
Fix slack bot form + LLM provider form
gmail refactor + permission syncing (gmail refactor + permission syncing onyx-dot-app/onyx#3021)
add default api keys for cloud users (add default api keys for cloud users onyx-dot-app/onyx#3044)
ensure we reset all (Ensure we reset all onyx-dot-app/onyx#3048)
add to async get session as well (add to async get session as well onyx-dot-app/onyx#3046)
Improve background token refresh
form
Clean up tooltips (Clean up tooltips onyx-dot-app/onyx#3047)
ensure props aligned (ensure props aligned onyx-dot-app/onyx#3050)
update
k
clarity
add sentry to spawned indexing task
limit session scope of index attempt (use id's where appropriate as w… (limit session scope of index attempt (use id's where appropriate as w… onyx-dot-app/onyx#3049)
Create connector
fix stale indexing tasks being allowed to run after a restart
comment why we need sentry here
need to see vespa url for container debugging
evaluate None to default
Fix errors and cleanup
fix usage report pagination
functioning
update slack
updated cleanup
no typing
update slack
validated approach
minor udpate
remove print
update redirect
update persona defaults (update persona defaults onyx-dot-app/onyx#3042)
Google Drive Improvements (Google Drive Improvements onyx-dot-app/onyx#3057)
minor improvements (Certificate improvements onyx-dot-app/onyx#3081)
Evaluate None to default (Evaluate None to default onyx-dot-app/onyx#3069)
PG Dev Max Connections (PG Dev Max Connections onyx-dot-app/onyx#3082)
Clean up connector
wait for db before allowing worker to proceed (reduces error spam on … (wait for db before allowing worker to proceed (reduces error spam on … onyx-dot-app/onyx#3079)
Fix failure cases
Remove linux
Migrate tenant upgrades to data plane (Migrate tenant upgrades to data plane onyx-dot-app/onyx#3051)
adjust default postgres schema for slack listener (adjust default postgres schema for slack listener onyx-dot-app/onyx#3088)
account for malformatted starter messages
File Connector Metadata (File Connector Metadata onyx-dot-app/onyx#3089)
fix saving docker logs (fix saving docker logs onyx-dot-app/onyx#3090)
Reenable OpenAI Tokenizer (Reenable OpenAI Tokenizer onyx-dot-app/onyx#3062)
Remove ee (Remove ee onyx-dot-app/onyx#3093)
Robustify cloud deployment + include initial KEDA configuration (Robustify cloud deployment + include initial KEDA configuration onyx-dot-app/onyx#3094)
Silence auth logs (Silence auth logs onyx-dot-app/onyx#3098)
re-enable helm (re-enable helm onyx-dot-app/onyx#3053)
New assistants api (New assistants api onyx-dot-app/onyx#3097)
Cleaner EE fallback for no op (Cleaner EE fallback for no op onyx-dot-app/onyx#3106)
moderate component fixes (moderate component fixes onyx-dot-app/onyx#3095)
improved mobile scroll (Improved mobile scroll onyx-dot-app/onyx#3110)
Support LITELLM_EXTRA_BODY env variable (Support LITELLM_EXTRA_BODY env variable onyx-dot-app/onyx#3119)
doc sync celery refactor (doc sync celery refactor onyx-dot-app/onyx#3084)
Add referral source to cloud on data plane (Add referral source to cloud on data plane onyx-dot-app/onyx#3096)
Fix widget example
clean up logging in light worker (clean up logging in light worker onyx-dot-app/onyx#3072)
update organization (update organization onyx-dot-app/onyx#3118)
quick fix for google doc sync
limited role api keys (limited role api keys onyx-dot-app/onyx#3115)
Added support for page within a page in Confluence (Added support for page within a page in Confluence onyx-dot-app/onyx#3125)
Make curators able to create permission synced connectors (Make curators able to create permission synced connectors onyx-dot-app/onyx#3126)
add creator id to cc pair (add creator id to cc pair onyx-dot-app/onyx#3121)
Updated google copy and added non admin oauth support (Updated google copy and added non admin oauth support onyx-dot-app/onyx#3120)
use text()
scale indexing sql pool based on concurrency (scale indexing sql pool based on concurrency onyx-dot-app/onyx#3130)
Cloud improvements (Cloud improvements onyx-dot-app/onyx#3099)
minor clarity (minor clarity onyx-dot-app/onyx#3116)
Cohere (Cohere onyx-dot-app/onyx#3111)
unnecessary python setup
Fix Quotes Prompting (Fix Quotes Prompting onyx-dot-app/onyx#3137)
minor configuration updates (minor configuration updates onyx-dot-app/onyx#3134)
Improved tokenizer fallback (Improved tokenizer fallback onyx-dot-app/onyx#3132)
Bugfix/overlapping connectors (Bugfix/overlapping connectors onyx-dot-app/onyx#3138)
Add error message on JSON decode error in CustomTool
reworked drive+confluence frontend and implied backend changes (reworked drive+confluence frontend and implied backend changes onyx-dot-app/onyx#3143)
Feature/playwright tests (Feature/playwright tests onyx-dot-app/onyx#3129)
Fixed int test (Fixed int test onyx-dot-app/onyx#3148)
nits + pricing
Made external permissioned users and slack users show diff (Made external permissioned users and slack users show diff onyx-dot-app/onyx#3147)
Add back employee context injection
Add GTM tracking
Update login page

Description

[Provide a brief description of the changes in this PR]

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Accepted Risk

[Any know risks or failure modes to point out to reviewers]

Related Issue(s)

[If applicable, link to the issue(s) this PR addresses]

Checklist:

All of the automated tests pass
All PR comments are addressed and marked resolved
If there are migrations, they have been rebased to latest main
If there are new dependencies, they are added to the requirements
If there are new environment variables, they are added to all of the deployment methods
If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
Docker images build and basic functionalities work
Author has done a final read through of the PR right before merge

…eature/reset_indexes

* modal fallback * nit * k * k

* Fix structured outputs * Add back rest

* check for index swap * initial bones * kk * k * k: * nit * nit * rebase + update * nit * minior update * k * minor integration test fixes * nit * ensure we build test docker image * remove one space * k * ensure we wipe volumes * remove log * typo * nit * k * k

…ture/reset_indexes

* upgraded model strings * trolled * we do a little trolling * reeeeeee * alembic upgrade * added ignore * bump litellm * k * nit --------- Co-authored-by: pablodanswer <[email protected]>

* check for index swap * k * minor * k * nit

* add posthog + layout rework * remove posthog node * nit

* ensure proper sentry silencing * add comments

* remove rt * nit * add minor alembic revision * functional migration * replace usp * k * typing

* add tenantJWTSrategy * nit

…ot-app#2932) * Move user fetching to SS * Cleanup * Add more logging * Small cleanup

* feat: add clean logging for api routes * nit * `MULTI_TENANT` must be shared config * nit

* temporary stash * welcome flow * minor update * k * minor updates to welcome flow

* in progress PoC * working limited user, needs routes to be marked next * make selected endpoint available to limited user role * xfail on test_slack_prune * add comment to sync function --------- Co-authored-by: Richard Kuo <[email protected]>

…pp#3126) * Make curators able to create permission synced connectors * removed editing permission synced connectors for curators * updated tests to use access type instead of is_public * update copy

* add creator id to cc pair * fix alembic head * show email instead of UUID * safer check on email * make foreign key relationships optional * always allow creator to edit (per hagen) * use primary join * no index_doc_batch spam * try this again --------- Co-authored-by: Richard Kuo <[email protected]>

) * Updated google copy and added non admin oauth support * backend update * accounted for oauth * further removed class variables * updated sets

* add improved cloud configuration * fix typing * finalize slackbot improvements * minor update * finalized keda * moderate slackbot switch * update some configs * revert * include reset engine!

* add cohere default * finalize * minor improvement * update * update * update configs * ensure we properly expose name(space) for slackbot * update config * config

use text()

…ython_setup unnecessary python setup

* silence warning * improved fallback logic * k * minor cosmetic update * minor logic update * nit

* fix tenant logging * upsert only new/updated docs, but always upsert document to cc pair relationship * better logging and rough cut at testing

…dot-app#3143) * reworked drive+confluence frontend and implied backend changes * fixed oauth admin tests * fixed service account tests * frontend cleanup * copy change * details! * added key * so good * whoops! * fixed mnore treljsertjoslijt * has issue with boolean form * should be done

* initial PoC * preliminary working config * first cut at chromatic tests * first cut at chromatic tests * fix yaml * fix yaml again * use workingDir * adapt playwright example * remove env * fix working directory * fix more paths * fix dir * add playwright setup * accidentally deleted a step * update test * think we don't need home.png right now * remove unused home.png --------- Co-authored-by: Richard Kuo <[email protected]>

…app#3147) * Made external permissioned users and slack users show diff * finished * Fix typing * k * Fix * k --------- Co-authored-by: Weves <[email protected]>

Signed-off-by: Alex Co <[email protected]>

rkuo-danswer and others added 30 commits October 25, 2024 09:35

Merge branch 'patch-1' of https://github.com/Yash-2707/danswer into f…

84d551e

…eature/reset_indexes

Ensure proper modal fallback (onyx-dot-app#2906)

76415af

* modal fallback * nit * k * k

Add pagination & Remove req.obj from connectors.tsx

a348caa

Fix structured outputs (onyx-dot-app#2923)

bd63119

* Fix structured outputs * Add back rest

Merge branch 'main' of https://github.com/danswer-ai/danswer into fea…

0ed77aa

…ture/reset_indexes

upgraded claude model strings (onyx-dot-app#2876)

94edcac

* upgraded model strings * trolled * we do a little trolling * reeeeeee * alembic upgrade * added ignore * bump litellm * k * nit --------- Co-authored-by: pablodanswer <[email protected]>

check for index swap (onyx-dot-app#2922)

5e01d6b

* check for index swap * k * minor * k * nit

Fixed polling issue with timezone

8023caf

add posthog + layout rework (onyx-dot-app#2926)

9def9f0

* add posthog + layout rework * remove posthog node * nit

Backport Test (onyx-dot-app#2930)

8c2138a

Backport Branch Fix (onyx-dot-app#2931)

c0ab86b

feat: sentry updates (onyx-dot-app#2929)

848d86b

Backport Test (onyx-dot-app#2935)

ccc68c5

Backport Auth (onyx-dot-app#2936)

b4bc6d9

ensure proper sentry silencing (onyx-dot-app#2934)

e8c28e7

* ensure proper sentry silencing * add comments

Backport Permissions (onyx-dot-app#2937)

80f8d7a

Harmless Backport Test (onyx-dot-app#2938)

4bcd65e

Backport Test (onyx-dot-app#2939)

accdd58

Backport Test (onyx-dot-app#2940)

f6f9112

Backport Correct Branch (onyx-dot-app#2941)

e6bef57

Backport Test Final (onyx-dot-app#2942)

aa0f307

remove rt + home-grown sitemap parsing (onyx-dot-app#2933)

088551a

* remove rt * nit * add minor alembic revision * functional migration * replace usp * k * typing

Tenant aware JWT strategy (onyx-dot-app#2943)

1261d85

* add tenantJWTSrategy * nit

Move user fetching to SS + parallelize some server-side calls (onyx-d…

0c2cc74

…ot-app#2932) * Move user fetching to SS * Cleanup * Add more logging * Small cleanup

Clean up code

e5d84ca

Feat: add clean logging for api routes (onyx-dot-app#2928)

da3c5e3

* feat: add clean logging for api routes * nit * `MULTI_TENANT` must be shared config * nit

Refactor & cleanup code, process tickets in batches

e5494f9

a (onyx-dot-app#2815)

a1bfa78

Onboarding nits (onyx-dot-app#2907)

179dc41

* temporary stash * welcome flow * minor update * k * minor updates to welcome flow

rkuo-danswer and others added 28 commits November 13, 2024 16:15

Added support for page within a page in Confluence (onyx-dot-app#3125)

60471b6

Make curators able to create permission synced connectors (onyx-dot-a…

a50a394

…pp#3126) * Make curators able to create permission synced connectors * removed editing permission synced connectors for curators * updated tests to use access type instead of is_public * update copy

Updated google copy and added non admin oauth support (onyx-dot-app#3120

485f3f7

) * Updated google copy and added non admin oauth support * backend update * accounted for oauth * further removed class variables * updated sets

use text()

65a205d

scale indexing sql pool based on concurrency (onyx-dot-app#3130)

d68f8d6

Cloud improvements (onyx-dot-app#3099)

facf1d5

* add improved cloud configuration * fix typing * finalize slackbot improvements * minor update * finalized keda * moderate slackbot switch * update some configs * revert * include reset engine!

minor clarity (onyx-dot-app#3116)

6e91964

Cohere (onyx-dot-app#3111)

7c84105

* add cohere default * finalize * minor improvement * update * update * update configs * ensure we properly expose name(space) for slackbot * update config * config

Merge pull request onyx-dot-app#3131 from danswer-ai/bugfix/session_text

21d5cc4

use text()

unnecessary python setup

54dc1ac

Merge pull request onyx-dot-app#3135 from danswer-ai/bugfix/helm_ct_p…

637b6d9

…ython_setup unnecessary python setup

Fix Quotes Prompting (onyx-dot-app#3137)

97932dc

minor configuration updates (onyx-dot-app#3134)

ddff7ec

Improved tokenizer fallback (onyx-dot-app#3132)

24be13c

* silence warning * improved fallback logic * k * minor cosmetic update * minor logic update * nit

Bugfix/overlapping connectors (onyx-dot-app#3138)

7015e6f

* fix tenant logging * upsert only new/updated docs, but always upsert document to cc pair relationship * better logging and rough cut at testing

Add error message on JSON decode error in CustomTool

259fc04

Fixed int test (onyx-dot-app#3148)

618bc02

nits + pricing

521425a

Made external permissioned users and slack users show diff (onyx-dot-…

954b5b2

…app#3147) * Made external permissioned users and slack users show diff * finished * Fix typing * k * Fix * k --------- Co-authored-by: Weves <[email protected]>

resolve conflicts

e4df35e

Signed-off-by: Alex Co <[email protected]>

Add back employee context injection

df6463f

Signed-off-by: Alex Co <[email protected]>

Add GTM tracking

58c030a

Signed-off-by: Alex Co <[email protected]>

Update login page

7806de7

Signed-off-by: Alex Co <[email protected]>

backport search logic

74ac5b3

Signed-off-by: Alex Co <[email protected]>

onimsha merged commit 144a9dd into main Nov 27, 2024
0 of 7 checks passed

onimsha deleted the chore/merge-upstream-2024112601 branch November 27, 2024 04:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore/merge upstream 2024112601 #79

chore/merge upstream 2024112601 #79

onimsha commented Nov 27, 2024

chore/merge upstream 2024112601 #79

chore/merge upstream 2024112601 #79

Conversation

onimsha commented Nov 27, 2024

Description

How Has This Been Tested?

Accepted Risk

Related Issue(s)

Checklist: