add further control for metadata update

update readme with new information

README.md

@@ -6,6 +6,9 @@ It is finally here. There is now a way to search all of Myrient's offerings.
 - 800MB-ish of memory for running the server
 - HTTPS for some CORS functions to work correctly.
 
+# Post Metasearch update
+Please clear your elasticsearch instance, and possibly run a new file rebuild to ensure there are no errors after updating your docker container or pulling the repo.
+
 # Self-Hosting
 
 ## Docker Method (Recommended)
@@ -69,6 +72,12 @@ server {
 For the SSL certificate you can use certbot via the `certbot -d servername.tld` command and adding it to your `crontab`.
 [Additional Information for Certbot Setup](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04)
 
+# Opengraph
+To ensure OpenGraph metadata embed for chat apps works correctly, please be sure to set `HOSTNAME` in `.env` or `docker-compose.yml` to the FQDN (fully qualifed domain name) of the server that is hosting the site.
+
+# Metadata
+To enable metadata synchronize and matching, you will need to create a developer application in the [Twitch TV Developer Console](https://dev.twitch.tv/console) and then add your client id to `TWITCH_CLIENT_ID` in `.env` or `docker-compose.yml` along with adding your client secret to `TWITCH_CLIENT_SECRET`. Metadata takes about half an hour to synchronize from IGDB to your database, and about another half an hour to match via Postgres Full Text Search. Once all other database maintenance operations are done, the database will attempt to match anything that still isn't matched using a much slower fuzzy trigram search that can take up to a day to complete. These processes won't run again until a new crawl of myrient has been performed and the file count has increased.
+
 # Contributing
 You know the usual fluff.  
 Is there a missing category or string association? `lib/categories.json` and any of the files under `lib/json/relatedkeywords` can both updated to include these. If you do update/improve these, please put in a pull request so that it can be added to the public hosted server, as well.  

lib/metadatasearch.js

@@ -231,7 +231,6 @@ export default class MetadataSearch {
         retryCount = 0;
       }
       console.log(`\nFinished syncing metadata in ${timer.elapsed()}`);
-      await this.matchAllMetadata();
     } catch (error) {
       if (error.code === "ERR_BAD_REQUEST" && !retrying) {
         this.setupClient();

server.js

@@ -76,6 +76,7 @@ let metadataSearch = new MetadataSearch();
 
 async function getFilesJob() {
   console.log("Updating the file list.");
+  let oldFileCount = fileCount || 0
   fileCount = await getAllFiles(categoryList);
   if (!fileCount) {
     console.log("File update failed");
@@ -86,9 +87,14 @@ async function getFilesJob() {
   if(await Metadata.count() < await metadataSearch.getIGDBGamesCount()){
     await metadataSearch.syncAllMetadata();
   }
+  if(fileCount > oldFileCount){
+    await metadataSearch.matchAllMetadata()
+  }
   await optimizeDatabaseKws();
   //this is less important and needs to run last.
+  if(fileCount > oldFileCount){
     metadataSearch.matchAllMetadata(true)
+  }
 }
 
 function buildOptions(page, options) {