Needs for IEEE VR 2020 #29
Comments
|
for the webpage refresher, would it be sufficient to auto-refresh after a fixed duration? we might be able to get away with having this be a server-wide setting of auto-refresh rate for web page thumbnails. |
|
AWS lambda is sufficiently cheap, now that I've run the math (please check :)) Lets say you had 100 web page thumbnails across your rooms, and you had 40 hours/week of active CCU viewing those thumbnails. for a 30s refresh interval this means 100 thumbnails * (40 hours * 120 refreshes per hour) = 480000 refreshes a week. if they take say 10s to run with 3gb memory each that's 14.5M gb/s a week, so 58 gb/s a month. lambda costs $0.20 per million requests and 0.000016667 per gb/s so the total cost of that would be a $0.000016667 * 58 gb/s = $0.001 for CPU and an additional $0.40 for requests, so less than $0.50/mo :) And that's for 100 concurrently 'viewed' thumbnails, which is relatively high. |
|
Those numbers are mind boggling (that it can be so small). For the VR / conference use case, yes, fixed update is plenty good. I would imagine that something like 1 min or 30s intervals while the conference is active, and cranking it down when it's not, would be fine.
I think any web content that changes faster than each 30s or so probably needs to be handled differently (e.g., have it be fetched and rendered locally, as we're looking at separately). |
|
Yup the refreshes themselves would be dependent upon room presence (basically, we'll have a client promote themselves to the 'refresher', similar to what we do for videos, which have an implicit 'timekeeper'.) So you'd only be paying for refreshes when people are actually looking at them, so it'd be self-regulating (and arguably much cheaper than the hypothetical outlined above, since that assumed not just 40 hours/wk of CCU, but basically 40 hours/wk of "eyeballs who can see 100 thumbnails") |
|
The refresh-on-demand case would still be covered by the existing refresh button we have now. So this would be probably a global site-wide switch (off by default, and only available to AWS stacks since the lambda is available) to enable auto-refresh for all thumbnails. |
|
OMG, I totally screwed up the units here. Your intuition is right, that was way too cheap. This seems to be a non starter actually unless we think my hypothetical is wrong. The AWS pricing is in gb/s, not Mgb/s |
|
So, $500/month for that example? or $500000? |
|
Let me try this again (with less confidence now I'm able to do arithmetic anymore :):
In the case above, you'd be doing approx 2M thumbnail fetches per month, so it'd cost ~$1000/mo in CPU and a few cents in requests. For the conference case, if you had 20 rooms with 2 thumbnails each that's 40 (not 100) thumbnails, but it would still be a few hundred bucks a month if you expected at least one person to be in each of those rooms for 40 hours a week (not far fetched.) So this seems probable a non-starter, unless we dial the refresh duration up significantly. If it was 5 minutes instead of 30 seconds, then the cost for the hypothetical above drops by an order of magnitude to $100/mo. |
|
So, what about the idea of just having a discord bot command, either per room or globally? Perhaps with the corresponding in-room command (that only admins can do, probably)?
|
|
doing a global effect on all rooms from discord is relatively uncharted territory -- i'd be nervous estimating what'd be involved there since it would be the first case of us doing something like "walk all the rooms in a set and apply some transformation to room state." having an in-game mechanic seems fine, the chat command could be supplemented by adding the "refresh" button to the object menu as well (when you click on the object in the object list.) however I thought the idea here was that we don't 'trust' that anyone in the room would be hitting these refresh buttons (either because there are too many to hit, or we don't expect a trusted 'moderator' to be present in the rooms to do so.) So if that's the case we either need a feature to walk all the rooms to do it (discord or otherwise) or we need to have it be based upon a refresh duration/expiration. |
|
we may be able to significantly reduce the AWS lambda costs of a higher refresh rate if we slightly adjust the contract here. basically, if we can determine the content hasn't changed, we can avoid the lambda function call. what this would mean is that when the client attempts an auto-refresh, it first uses HTTP caching protocols (eg an ETag) to determine if the current thumbnail is stale or not, before requesting a full re-fetch. That seems likely to work, but ofc there's always a risk that this could spiral out of control into those costs above if the content always appeared stale due to unexpected cache misses. (We could potentially have a rate limiter on this to prevent that, but complexity at that point would be getting pretty heavy.) |
|
Ideally, doing it fully automatically is what I'd want. But with a really slow update, that may be less useful, and a high update may be cost prohibitive. Using cache's would be a good idea, but I worry about the tradeoff of work vs benefit. The only thing that's a complete non-starter is "Go into the room and do a refresh on the object with the 3D UI" ... too slow, given the friction to get in and navigate. Even "Go into the room and run /refresh_pages" is probably too slow. But, having 30 or 40 channels in discord, where we go an past "!hubs refresh_pages" in each is not unreasonable, esp if that's easy to implement. Having a corresponding in-room / command would be good too. |
|
I think doing a global effect is going to be tough to implement, and subject to a lot of risk as well. Basically, the discord bot would need to start having side effects in the rooms, vs just being a passive observer. I think there's a lot of risk to that kind of thing, since a bug or runaway process etc could cause a global failure. (And in the case of hub's discord bot, a global failure across all connected discord servers becomes a risk too) |
|
We are talking about the same thing, right? I’m thinking of just affecting the room connected to one discoed channel; let’s ignore “all”.
Is an in-room command like “/refresh_pages” (the pages for that room) not doable either?
…---
Blair MacIntyre
Principal Research Scientist
snt frm iOS kybrd, pls pardon typos and weird auto-corrections
blairmacintyre.me && pronoun.is/he/him
________________________________
From: Greg Fodor <[email protected]>
Sent: Wednesday, February 5, 2020 2:06 PM
To: mozilla/hubs-cloud
Cc: Blair MacIntyre; Author
Subject: Re: [mozilla/hubs-cloud] Needs for IEEE VR 2020 (#29)
I think doing a global effect is going to be tough to implement, and subject to a lot of risk as well. Basically, the discord bot would need to start having side effects in the rooms, vs just being a passive observer. I think there's a lot of risk to that kind of thing, since a bug or runaway process etc could cause a global failure. (And in the case of hub's discord bot, a global failure across all connected discord servers becomes a risk too)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#29>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAKJJ7NGWXRTPYCM342XTF3RBME2PANCNFSM4KOKG5DQ>.
|
|
Oh I see. So you'd still iterate over each room in discord to do it. That makes it less scary, but is still uncharted territory and a bit risky since the discord but would become a more active agent in that room. Basically anywhere we are doing "writes" in the discord bot raises red flags, since the blast radius of a bug in that code is extremely wide. (Most of our server side code is naturally isolated, since we don't have any global, cluster-wide contexts due to the way elixir processes + state are organized) |
|
Ie, the discord bot, oddly enough, is one of the few places across all our code where you actually have the leverage to easily affect multiple rooms, and all connected users of those rooms. (Via a |
|
I think this feature is valuable enough globally since this solves a lot of the "we want a web page in hubs" use cases. I could imagine us doing a nice job with it, and making it more of a server-side 'push' concern. Basically, when there is a thumbnail in a room, we could spawn a dedicated elixir process to manage it. (These are cheap.) It would periodically fetch the origin looking for changes, and could incorporate cluster-wide rate limiting to ensure costs are capped across all thumbnails. The net effect would be the server itself could notify the room participants via a 'push' to the channel that the thumbnail is stale, and pre-fetch + cache the new version. |
|
Since the server is doing this, the "time to change detection" could be dialed down to only a few seconds, depending on how aggressively we want to poll the origin. And then we could (locally, per thumbnail) rate limit the actual refreshes. So the idea would be, you would only see at most one change per N seconds, but if a change occurs, you will see it within M seconds of the change actually happening assuming N seconds have passed. |
|
eg, a 15s polling interval and a 5m refresh limit would eliminate the need for global rate limiting altogether. if you made a change at the origin, you'd see it in at most 5m across all clients, but if enough time has passed since the last update, you'd see it in 7s on average. (plus the thumbnail generation time of a few seconds) |
|
This seems like something we could turn on by default and not even expose as an option actually under that setup. |
|
Also this would be a 'big project' so not something I'd expect us to turn around quickly. But the writeup above seems at least like the 'right' solution to this problem. For the IEEE case, we probably should plan on some other mechanic tho. |
|
This also might be orthogonal to the real need here; the ideal solution is just having content be updated continuously inside the system, not using HTML. So, a component that is generating a texture to update an image so it reflects everywhere. Perhaps this can be what we experiment with for custom components. I was actually imagining that this happens locally on all clients, rather than being synchronized. |
|
yeah that's a different issue, that cuts into component extensibility imo |
|
yes. I was pondering that a fast/frequent refresh is so expensive it might be best not to implement in a way that people can abuse it. For content that changes frequently, such as the twitter feed mocked up in the Hello-WebXR demo the team did, it might be better to implement that entirely differently. For example, a "ScriptBot" might be attached to a room, and run scripts that update certain images. Or, a custom extension may synchronize the base data (e.g., the hashtag being displayed on the twitter panel) such that anyone can "change it" (somehow), but it may fetch and render locally in each client. Any of these could be implemented as web pages, but (as you suggest) the cost might be prohibitive. |
|
Yeah, another approach to that in a similar spirit would be an installable app that used WebRTC and a headless browser context to just render web pages via video into a room. There's already a PoC of this put together by someone in the aframe community. |
|
for some uses that might be fine; I'm imagining the case where there are lots of little 2D bits around a room, that might be hard to pull off that way. |
|
Since we have the list of issues tracked independently, I'm going to resolve this. Audio override by room owners is captured in Hubs-Foundation/hubs#1853 Dynamic web content is captured in Hubs-Foundation/hubs#2424 and Hubs-Foundation/hubs#1688 |
I wanted to drop a message here where I can track the things I need working for IEEEVR. I'll likely add to this list as I understand it.
The text was updated successfully, but these errors were encountered: