Planning

[UP2040499]

Once Intelligence Statement and source(s) are input:

• Semantic analysis conducted on statement
  • Determines the human emotional reaction to the statement.
  • Polarised emotions resultant from the statement, means the statement is more of a headline, and is unlikely to be a reliable source of intelligence
  • Nonetheless, this should not rule the statement out as this can give the user an idea of potential disinformation campaigns.
• Running the statement on Google search
  • Simple search could produce corroborating sources
  • These sources would have to be dissimilar to the sources used to create intelligence statement.
    • Can be achieved by comparing similarity and having a similarity threshold (e.g., do not accept sources that have similarity to original sources of >20%)
  • Sources could be of different formats, e.g., a social media post vs a news article
  • Key information would have to be gathered from each potential source
    • Can be done using BERT QA
• Search social media sites such as Reddit or Twitter
  • Like above, sources must be dissimilar to original source
  • Can produce more up-to-date results
  • Higher likelihood of misinformation
• Semantic analysis conducted on potential corroborating sources
  • Would be done on the headlines from the potential corroboration
  • Particularly useful for social media posts which are likely shorter length than news articles.
• Sources accepted by the tool using the above will then be ran through the BERT QA model
  • This will assign highest priority to documents that mention specific names or places
    • A source that details key information such as this will go to the top of potential corroboration list.
    • This is our 'target information' - information that is mentioned in original intelligence statement.
  • Assign higher priority to documents that mention entities (places, names, organisations, etc.) that appear more often in multiple sources.
    • Like above but will likely move multiple sources higher in the potential corroboration list.
    • However, a metric for gauging priority will be needed to give points to sources that mention popular entities. This is so that each document has an independent score and can get points for mentions of key information.
    • This is 'popular information'.
  • Mentions of locations should be put through geocoding to find exact geo coordinates, and nearby locations of interest (towns, cities, etc.).
    • Nearby locations should be checked to see if they match any locations in the target information.
  • Perhaps, by breaking down the target information to pull out only particular entities, we can use these entities to compare to
    • Try to prompt the user to include known associates and other relationships in the intelligence statements particularly those that could be unclear in open sources.
    • Intelligence statements should compile as much relevant information as possible. Should not just be "Person x did activity y in location z" rather, "Person x did activity x in location z. Person A is a known associate of person x, they both work for company B".
    • This gives the tool information that is unlikely to be gained through searches and can help widen the search domain. For example, in the second statement, the tool knows of two individuals who are of interest and will look for information that can support the statement that activity Y took place in location z. This relevant information may include both or just one of these individuals.
    • Relationships are much harder for the tool to automatically recognise, for example, just because person A is Facebook friends with person B does not mean they are associates or that person B would help person A in activity Y. However, being Facebook friends could support the intelligence statement that they are associates.
  • So, in summary, documents will be given a score for how many independent mentions of 'target information' and then given a lower score for each independent mention of 'popular information'.
    • Repetitions of target or popular information within the same source does not increase the score.
    • Scores should try to represent the importance of target information vs popular information.
      • This could assign a higher score to target information if the user wants more emphasis on sources that contain that target information
      • Or could assign higher scores to popular information if the user wants a bigger picture of information related to the statement. Can also be done when the intelligence statement does not contain much information on specific entities (names, places, organisations, etc.).
      • Target information scores should always be higher than popular information scores
      • For example, you could assign each independent mention of target information a score of +10 and each independent mention of popular information a score of +5. This would represent that popular information (widely disseminated) is half as important as the target information.
• Any mention of sensitive or closed source intelligence sources can be ignored – mostly because there is no way to validate these sources (using open sources).
  • This could be changed if the user has high confidence in these classified sources. This would be an overriding bias and the tool would simply show any open sources that corroborate information from these sources.
  • By showing this as part of the 'evidence' alongside corroborating sources, semantic analysis and the overall confidence score for the intelligence statement

[UP2040499]

See SharePoint folder here for full design diagram(s).