-
Notifications
You must be signed in to change notification settings - Fork 1
Technical Challenges
These are some of the ways this repository works around the various technical limitations posed by the problem of searching the Points Shop.
Steam has thousands of apps, which can include both games and other virtual goods such as DLC. In fact, if you made a Steam API request for each app, you would hit the daily request limit before managing to finish querying every app. Even without this request limit, a program having to make thousands of web requests will take a large number of computing resources, which could result in the program crashing.
The Game Showcase provided by the Steam Card Exchange lists all apps that have obtainable cards or badges, which by extension indicates which apps have potential Points Shop items associated with them. In early 2022, this list contains about 11000 items, which is a considerably smaller data set with the guarantee that the Steam API will have Points Shop information about these apps.
Of course, this heavily relies on the Steam Card Exchange correctly and consistently updating their Showcase page with current app information, and could potentially be omitting apps exclusively listed on the Background Viewer, although this has yet to be confirmed. Thus, it is entirely plausible that the data set in this repository is only a partial percentage of the total Points Shop items provided by the Steam API.
Since the Steam API returns a maximum of 1000 items in one response, a cursor parameter is required to access further data in sets of 1000. On initial inspection, the correct approach might be to just have logic that processes each page of item data from https://api.steampowered.com/ILoyaltyRewardsService/QueryRewardItems/v1/ and would also account for newly added Points Shop items.
The problem with the QueryRewardItems endpoint provided by the Steam API is that it only provides the app ID that each item belongs to, and not the "friendly name" of the app itself (e.g. Items belonging to app ID 440 don't explicitly state that they belong to Team Fortress 2). To get the mapping of app name to app ID, one would need to access this information from https://api.steampowered.com/ISteamApps/GetAppList/v2/ which has its own caveats as well (see "Steam API notes").
Nonetheless, this repository still includes a mechanism for obtaining Points Shop information from these endpoints in this cursor-based way, and can be run using npm run start-recursive.
With the current approach using the Game Showcase provided by the Steam Card Exchange, apps exclusive to the Points Shop are accounted for and the resulting data is such that users are less likely to stumble across items from the Points Shop which are no longer purchasable.
If you think about the many third-party websites that extensively use the Steam API such as the Steam Card Exchange and backpack.tf, these have proper server infrastructure behind them to ensure those sites can handle high user loads along with a proper DBMS to store data. To pursue that kind of a setup for this repository would involve ongoing overheads in terms of time, money and maintenance.
Given the experimental nature of this repository, GitHub Pages is a viable hosting platform that is free and relatively easy to deploy, as long as everything can run as a static web page. So as long as all the Steam API data can be comfortably loaded into memory (i.e. don't use Internet Explorer), the core searching functionality can be provided with impeccable speed.
Note that since all the logic is on the client-side, a user could technically crash their own web browser by manually editing the web page to remove the relevant validation checks and then search for a short, generic word like "a".
Given that the search results provide a link to the item's unanimated image, the next logical progression would be to show the actual image. However, this has a number of implications on both the complexity of this repository and on the Steam image hosting servers as well, partly dependent on the specific method in which this feature is implemented:
Hotlink image files - While an image can be directly loaded from the source into the search results page, it's generally understood as a bad practice in a variety of ways, so it makes sense that it should be avoided, even for a simple application such as this one.
Download all images and load them from local storage - Another idea is to download every single item image from Steam and show those instead of using Steam's network resources directly. This brings about major scalability and maintainability concerns, particularly around the recommended repository size limits for a GitHub repository.
Upload all images to a third party image hosting service and hotlink to them - While this does work around the repository limitations imposed by GitHub, some of these services have their own terms and conditions around how long images will be hosted for, and especially around how that service is intended to be used. For example, Imgur's FAQ explicitly states that it is not to be used as a content delivery network, which is more or less what's being done in this case.
While not entirely ideal, simply showing a link to the item image should be sufficient enough to still be considered useful to an end user. As a beneficial side effect, this also means users won't have to see (potentially NSFW) images they'd rather not look at unless they deliberately click on the link provided.