Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimise SOX #391

Closed
5 tasks done
shu8 opened this issue Apr 25, 2019 · 1 comment
Closed
5 tasks done

Optimise SOX #391

shu8 opened this issue Apr 25, 2019 · 1 comment
Assignees
@shu8
Labels
approved contributions welcome feature request
Milestone
v2.5.0

Comments

@shu8
Copy link
Member

shu8 commented Apr 25, 2019 •
edited
Loading

SOX can sometimes add up to a few seconds to page load times, and its performance seems to decrease as more features are enabled.

This causes users to have to think about which features they want most rather than enabling all that they like.

Whilst SOX inevitably will cause some increase in load times due to the changes/insertions it does to the DOM, I'm confident it can be improved.

I logged some very quick timing data on the time it takes from the first feature to load till the last using the features I have enabled (I have 50 out of the current 79 available enabled):

  • 216.962890625ms
  • 317.482177734375ms
  • 404.2578125ms
  • 197.465087890625ms
  • 527.699951171875ms

They seem to range from 200ms to 500ms. I'm not entirely sure how they're so different but it would be great to reduce this!

Some ideas on what I'm planning to do at first:

  • Go through SOX's use of jQuery and try to optimise inefficient selectors/replace them with vanilla JS. jQuery's docs and this tutorial mention some ways jQuery selectors can be improved
  • Cache DOM elements that have already been queried
  • Look into how efficient the Mutation Observers are now
    Many SOX features rely on Mutation Observers so they can react to changes in the DOM.
    This has led to SOX adding lots of different ones for different areas of the page, and I'm sure some of them could at least be made more specific.
    Alternatively have a single Mutation Observer for SOX which each feature hooks into to find changes it's interested in.
    Performance of MutationObserver to detect nodes in entire DOM is an interesting read
  • See if the Chrome Dev Tools Profiler can be of any use in optimising SOX
  • Start maintaining some sort of cache of API request results -- a very common browse on SE sites must definitely result in lots of very similar/same API requests (e.g. getting a user's details, or a post's revisions). Caching for even a minute or two might reduce the number of requests made

If anyone has any other ideas, please do comment here, or submit a pull request!
@shu8 shu8 added this to the v2.5.0 milestone Apr 25, 2019
@shu8 shu8 self-assigned this Apr 25, 2019
@shu8 shu8 pinned this issue Apr 25, 2019
shu8 added a commit that referenced this issue Apr 25, 2019
@shu8
Log time taken for each feature's execution and overall performance
f7e59c2 
Start of #391
shu8 added a commit that referenced this issue Apr 25, 2019
@shu8
Optimisation: Remove jQuery usage in markEmployees
33586fe 
Reduces execution time from ~4.6ms to ~2.5ms

Part of #391
shu8 added a commit that referenced this issue Apr 25, 2019
@shu8
Optimisation: Remove jQuery usage in highlightQuestions
c101f86 
Reduces execution time from ~3.6ms to ~2.3ms

Part of #391
shu8 added a commit that referenced this issue Apr 26, 2019
@shu8
Optimisation: Reduce jQuery usage in colorAnswerer
ea1664d 
Reduces execution time from ~1ms to ~0.6ms

Part of #391
shu8 added a commit that referenced this issue Apr 26, 2019
@shu8
Optimisation: Reduce jQuery usage in kbdAndBullets
a27ab4f 
Reduces execution time from ~1s to ~0.45s

Part of #391
shu8 added a commit that referenced this issue Apr 26, 2019
@shu8
Optimisation: Remove jQuery usafe in shareLinksPrivacy
ebc1a86 
Reduces execution time from ~4.5ms to ~3.3ms

Part of #391
shu8 added a commit that referenced this issue Apr 26, 2019
@shu8
Optimisation: remove jQuery usage in shareLinksMarkdown
bd8cf31 
Reduces execution time from ~1.5ms to ~1.1ms

Part of #391
@shu8 shu8 mentioned this issue May 25, 2019
Merged
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Use API for editReasonTooltip
5615d46 
I'm not sure why, but previously I was performing a GET request to
*each* post that had a revision and manually extracting the edit comment
from the HTML.

This was unnecessary because it a) required many more page requests that
needed and b) was less efficient as it needed to parse the HTML to
extract the revision comment.

The function now uses a single API query with a heavy filter to only get
the data that is required (post ID, revision comment, revision type).

Part of #391.
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Optimisation: Reduce jQuery usage in confirmNavigateAway
c060c4c 
Part of #391
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Use single API call for standOutDupeCloseMigrated
e23a0ac 
Previosuly, one API call was being made for *each* post that had a
closed status next to its title.

Thi was unnecessary because it required many more API requests than
needed, and so used up more of the the API quota and bandwidth.

The function will now make a single API call with all the IDs that have
a closed status.

Part of #391.
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Optimisation: Reduce jQuery usage in sortByBountyAmount; Cache selectors
dbbc606 
Part of #391
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Optimisation: Reduce jQuery usage in answerTagsSearch; Cache selectors
8221a99 
Part of #391
shu8 added a commit that referenced this issue Jun 1, 2019
@shu8
Optimisations: Cache DOM selectors where possible
caf5165 
Part of #391
shu8 added a commit that referenced this issue Jun 2, 2019
@shu8
Implement API caching
a166ff1 
Most API requests will now be cached for 3 minutes before they are
requested from the API again. Note: the choice of 3 minutes was fairly
abritrary and might need tweaking in future.

This is to try and reduce the number of useless API calls made, saving
bandwidth and should speed up some SOX features (as they will not need
to wait for a response from the API if the data is already saved
locally).

A major change is that `sox.helpers.getFromAPI` will now essentially
return the 'inner' `items` object, rather than the exact response from
the Stack Exchange API (which had extra 'metadata' such as quota usage,
backoff details, and error messages). This means the function will now
return only an array of items, which will include the cached items, if
applicable.

Caching will be enabled *by default*, as long as multiple IDs are
provided (in an array) and the `featureId` property is sent in the call
to `sox.helpers.getFromAPI`.

I've implemented the caching to be per-feature and per-endpoint. This
means if two different features request to the same endpoint for the
same ID(s), the cache will not be shared. This is because features use
their own specific API filters, so sharing the cache between features
would mean there is potential for the data to not be what was requested.

Part of #391.
shu8 added a commit that referenced this issue Jun 2, 2019
@shu8
Minor performance improvements
e186e0c 
The Chrome Dev Tools JavaScript Profiler revealed some features are
being run on pages they shouldn't.

Also simplified regex for extracting ID from URLs.

Store length of array before iterating so it doesn't need to be
calculated on each iteration.

Part of #391.
shu8 added a commit that referenced this issue Jun 2, 2019
@shu8
Optimisations: Improve usage of MutationObservers
86d9713 
Previously, SOX's usage of MutationObservers was very inefficient; all
of them were targeting document.body, rather than a more specific
element where the change would occur.

I've changed `sox.helpers.observe` to now *require* a target, and
changed all usages of the function to include a more specific target.

This should hopefully reduce the impact SOX has on the performance of
pages whilst browsing.

Part of #391.
shu8 added a commit that referenced this issue Jun 2, 2019
@shu8
Only run features on appropriate pages
4f5c5eb 
Adds relevant `exclude` and `match` properties to sox.features.info.json
for all features, so features only run on pages they were designed to
run on.

Part of #391.
shu8 added a commit that referenced this issue Jun 2, 2019
@shu8
Allow for customisable cache duration per-feature
4b88d39 
This will allow data that we know is less likely to change to be cached
for longer, reducing the number of unnecessary API API requests further.

Part of #391.
@shu8 shu8 mentioned this issue Jun 2, 2019
Closed
@shu8
Copy link
Member Author

shu8 commented Jun 4, 2019

With these changes, I've seen times go down to ones along these lines:

  • 126.412109375ms
  • 133.839111328125ms
  • 150.77001953125ms
  • 164.4970703125ms

I'm fairly sure I'm using the same SE page I used to get the original numbers in this issue, and it definitely does seem to have improved.

Whilst there may (will!) be more improvements to be made, I've finished the main things I wanted to do (cache API responses, improve MutationObservers), so I'm going to close this and release v2.5.0 to master.

@shu8 shu8 closed this as completed Jun 4, 2019
@shu8 shu8 removed the in progress label Jun 4, 2019
@shu8 shu8 unpinned this issue Jun 4, 2019
@shu8 shu8 mentioned this issue Sep 3, 2019
Merged
@shu8 shu8 mentioned this issue Nov 12, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shu8 shu8

Labels
approved contributions welcome feature request
Projects
None yet
Milestone
v2.5.0
Development

No branches or pull requests
1 participant
@shu8