Display Spur data on IPInfo infobox
Open, MediumPublic
Actions

Assigned To

Authored By

	Niharika
	Jul 7 2023, 10:18 PM

Description

Motivation

As a user investigating temporary user contributions, I want to see reliable data about their IP address.

This ticket outlines which data in IP Info will be fetched from which data source. This ticket intends to serve as the authoritative source on which data will be displayed in IP Info after Spur integration.

Mock

IP-Info-box(default).png (552×932 px, 50 KB)

Figma design file

Data breakdown

Field	Source	Help text
Location	Maxmind	The geographic location associated with the IP address.
ISP	Maxmind	The Internet Service Provider associated with the IP address.
ASN	Maxmind	The autonomous system number associated with the IP address.
Organization	Maxmind	The organization operating this IP address.
Version	Parsed by MediaWiki	IP address version v4 or v6
Behaviors	Spur - client.behaviors	Specific types of activity devices on this IP Address have engaged in or have been previously affiliated with.
Risks	Spur - risks	Risks that Spur has determined based on their collection of data. These risks vary and will matter differently based on specific use-cases of Spur Context data.
Connection type	Spur - client.types	The different type of client devices that have been observed on this IP address.
Tunnel operator	Spur-tunnels[].operator	The provider or operator of the VPN service.
Proxies	Spur - client.proxies	Proxies that have been observed on clients using this IP address. This does not implicate all traffic from this IP Address to this proxy network, but rather the presence of a proxy on this IP Address and the ability for it to be used as a proxy.
Users on this IP	Spur - client.count	The average number of clients that have been observed on this IP address by Spur. It takes into account all activity from this IP address. This is calculated over a 24 hour period.
Active blocks	On-wiki data	The number of active blocks against this IP address.
Contributions	On-wiki data	The number of wiki edits made from this IP address.

Useful links

MaxMind documentation
Spur documentation - see "Fields" tab

Acceptance criteria

The infobox displays data as tabulated above, using the correct fields, sources and help text

Details

	Subject	Repo	Branch	Lines +/-
	Display IPoid data on IPInfo infobox	mediawiki/extensions/IPInfo	master	+166 -15

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
In Progress		Niharika	T324492 Temporary accounts - MVP
Open		None	T340895 [Epic] IP Info accommodations for temporary accounts
Open		STran	T341395 Display Spur data on IPInfo infobox
Resolved		Tchanders	T349723 Return Spur data from the IPInfo APIs
Resolved		kostajh	T354359 Update ProductionServices.php with proxy address for ipoid
Open		None	T356105 Instrument data sources used in IPInfo
Declined		None	T340628 Create mock provider for ipoid service requests
Resolved		STran	T339330 Update ENV variables to match expected production variables
Invalid		None	T339286 Allow IPInfo's infobox widget to display data from more than one data source
Resolved		kostajh	T339284 Deploy ipoid
Resolved		kostajh	T340984 Fix problems with data import
Resolved	BUG REPORT	• AGueyte	T340178 "Column 'anonymous' cannot be null"
Declined	BUG REPORT	• TThoabala	T340176 "Data too long for column 'behavior' at row 1"
Declined	BUG REPORT	None	T339837 "Data too long for column 'anonymous' at row 1"
Resolved	BUG REPORT	• TThoabala	T339359 'Parameter at position 2 is undefined\n' if tunnels.type is not set
Resolved	BUG REPORT	• AGueyte	T339345 "Duplicate entry '0' for key 'behavior'"
Resolved	BUG REPORT	• TThoabala	T339343 "Column 'type' cannot be null"
Declined	BUG REPORT	• TThoabala	T341605 Ipoid: ER_DATA_TOO_LONG
Declined	BUG REPORT	• AGueyte	T339325 "Duplicate entry '0' for key 'proxy'"
Invalid	BUG REPORT	None	T339324 "Data too long for column 'type' at row 1"
Invalid		None	T345107 Update Tunnels Type schema VARBINARY value
Resolved	BUG REPORT	• TThoabala	T339073 Density imported incorrectly
Resolved		STran	T344749 Refactor import-db and update-db into one script
Resolved		STran	T341122 Implement daily data update routine
Resolved		Tchanders	T344517 Fix statements generated for updated IP data
Resolved		Tchanders	T344520 Remove temporary files to avoid taking up unnecessary disk space
Resolved		STran	T344272 Use batching for daily data import
Resolved		STran	T325630 Implement call to data vendor
Resolved		Tchanders	T344499 Move behaviors/proxies/tunnels processing into an independent script
Declined		None	T344939 Log when the lookup fails on attempting to INSERT to actor_data (because the proxy/behaviour doesn't exist)
Resolved		Tchanders	T344921 Ensure dependent SQL statements are all run in the same batch
Resolved		Tchanders	T344981 Map tables should not have null for any of their values
Resolved		STran	T345757 Create orchestration script for daily data update
Resolved		None	T345761 Turn scripts into modules that export
Resolved		STran	T345684 Implement batch tracking for imports
Open	BUG REPORT	Tchanders	T341660 Investigate non-deterministic behaviour of import
Resolved		kostajh	T341129 Setup spec.yaml
Resolved		jijiki	T341326 Deploy ipoid to staging on Kuberenetes
Resolved		kostajh	T339331 Prepare for initial data import on production servers
Resolved		STran	T344941 Respond to data update and import errors
Duplicate		None	T355429 data pipeline: Don't attempt to re-import current day if full import already existed
Resolved		STran	T345156 DBA audit of ipoid database
Duplicate		STran	T345237 Investigate: Recovering from skipped updates when pulling from the daily feed
Resolved		kostajh	T346861 [M] Write CronJob configuration
Resolved		jijiki	T350903 Rename FEED_API_KEY
Resolved		kostajh	T348477 Update mariadb library to latest version 3 release
Resolved	BUG REPORT	kostajh	T349056 [S] split: too many files
Resolved		kostajh	T349171 [M] Implement proxy configuration for kubernetes deployment

Event Timeline

Niharika triaged this task as Medium priority.Jul 7 2023, 10:18 PM

Niharika created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 7 2023, 10:18 PM

Frostly subscribed.Jul 7 2023, 11:32 PM

Niharika updated the task description. (Show Details)Jul 8 2023, 12:04 AM

Niharika updated the task description. (Show Details)Jul 8 2023, 12:38 AM

Niharika updated the task description. (Show Details)Jul 8 2023, 12:40 AM

Niharika updated the task description. (Show Details)Jul 8 2023, 12:46 AM

Screen mockup can be found in this figma file

Niharika updated the task description. (Show Details)Jul 14 2023, 9:48 PM

Niharika updated the task description. (Show Details)Jul 18 2023, 9:56 PM

JSengupta-WMF updated the task description. (Show Details)Jul 19 2023, 10:33 AM

KColeman-WMF updated the task description. (Show Details)Sep 26 2023, 4:52 PM

KColeman-WMF updated the task description. (Show Details)

KColeman-WMF subscribed.

Tchanders mentioned this in T347417: Stop storing unnecessary data in the ipoid database.Sep 26 2023, 5:25 PM

JSengupta-WMF unsubscribed.Sep 27 2023, 7:35 AM

KColeman-WMF updated the task description. (Show Details)Sep 28 2023, 11:17 AM

KColeman-WMF updated the task description. (Show Details)Sep 28 2023, 11:22 AM

Niharika renamed this task from [WIP] IP Info data sources to IP Info data list (Spur and MaxMind) .Oct 17 2023, 8:41 PM

Tchanders renamed this task from IP Info data list (Spur and MaxMind) to [Epic] Display Spur data on IPInfo infobox.Oct 25 2023, 2:42 PM

Tchanders updated the task description. (Show Details)

Tchanders renamed this task from [Epic] Display Spur data on IPInfo infobox to Display Spur data on IPInfo infobox.Oct 25 2023, 3:12 PM

Tchanders added a parent task: T340895: [Epic] IP Info accommodations for temporary accounts.

Tchanders added a project: Trust and Safety Product Sprint.Nov 13 2023, 3:39 PM

Tchanders edited projects, added Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)); removed Trust and Safety Product Sprint.Dec 4 2023, 6:00 PM

Tchanders edited projects, added Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)); removed Trust and Safety Product Sprint (Sprint Lute (11th Dec. - 22nd Dec. 2023)).Jan 8 2024, 2:14 PM

STran claimed this task.Jan 11 2024, 9:17 AM

STran moved this task from Priority Backlog to In Progress on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.

I've missed it if we came to a conclusion earlier but could we confirm our treatment of the values?

Specifically, behaviors can return FILE_SHARING and a very long list of possible services (see https://docs.spur.us/data-types?id=client-behavior-enums). It seems reasonable to translate "File sharing" but what should we do about the others? eg.

	"behaviors": [
		"TOR_PROXY_USER",
		"FILE_SHARING",
		"SMILE_VPN_USER",
		"WARP_VPN_USER",

Presumably, whatever answer we come to here can be used for proxies and tunnel operators, which return similar lists.

Connection types and risks have reasonably enumerable lists and can be translated as expected.

cc @JSengupta-WMF @KColeman-WMF

STran added a subscriber: JSengupta-WMF.Jan 11 2024, 12:04 PM

And an example of how a very used IP could look. Many of these attributes can return more than one result.

Change 989827 had a related patch set uploaded (by STran; author: STran):

[mediawiki/extensions/IPInfo@master] Display IPoid data on IPInfo infobox

https://gerrit.wikimedia.org/r/989827

gerritbot added a project: Patch-For-Review.Jan 11 2024, 1:11 PM

JSengupta-WMF unsubscribed.Jan 11 2024, 1:25 PM

Tchanders mentioned this in T325645: Implement security-api data in IPInfo widgets.Jan 11 2024, 2:09 PM

Tchanders moved this task from In Progress to Needs review on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 12 2024, 11:49 AM

Change 989827 merged by jenkins-bot:

[mediawiki/extensions/IPInfo@master] Display IPoid data on IPInfo infobox

https://gerrit.wikimedia.org/r/989827

Maintenance_bot removed a project: Patch-For-Review.Jan 12 2024, 6:31 PM

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.14; 2024-01-16).Jan 12 2024, 7:00 PM

Tchanders moved this task from Needs review to Needs QA on the Trust and Safety Product Sprint (Sprint Northumbrian smallpipes (8th Jan.‘24 - 19th Jan.'24)) board.Jan 15 2024, 9:01 PM

Questions perhaps for @KColeman-WMF.

Do we need to review the wording in the help text?

Do we want the lists of "Behaviors", "Proxies", etc. to always be alphabetically ordered? Currently I believe the order can vary. This might make comparing data for two IP addresses harder to do.

There is also an open questions from @STran in T341395#9452875.

Otherwise, I have no other concerns and will move this along.

Test environment: IP Info 0.0.0 (19f6b7f) 08:17, 19 January 2024. ipoid commit e0250e5f4cb6942c5ccbbddc299458f624ecc01c.

Tchanders closed subtask T349723: Return Spur data from the IPInfo APIs as Resolved.Jan 29 2024, 3:29 PM

Tchanders merged a task: T339279: [Epic] Display data from ipoid in IPInfo infobox.Feb 7 2024, 5:12 PM

Tchanders added subscribers: Tchanders, kostajh, Madalina, STran.

Tchanders added subtasks: T356105: Instrument data sources used in IPInfo, T340628: Create mock provider for ipoid service requests, T339330: Update ENV variables to match expected production variables, T339286: Allow IPInfo's infobox widget to display data from more than one data source, T339284: Deploy ipoid.Feb 7 2024, 5:16 PM

Tchanders closed subtask T339286: Allow IPInfo's infobox widget to display data from more than one data source as Invalid.Feb 7 2024, 5:18 PM

Tchanders mentioned this in T339286: Allow IPInfo's infobox widget to display data from more than one data source.

In T341395#9481364, @dom_walden wrote:

Questions perhaps for @KColeman-WMF.

Do we need to review the wording in the help text?

Do we want the lists of "Behaviors", "Proxies", etc. to always be alphabetically ordered? Currently I believe the order can vary. This might make comparing data for two IP addresses harder to do.

There is also an open questions from @STran in T341395#9452875.

Hi @dom_walden

The help text provided in the table above has already been reviewed by product/design.

With regards the question from @STran: I think it makes sense to display the values as they are provided on Spur, expect when it's obvious what the ideal translation is. If we are including a line about the data source (as per design) then perhaps we can link to the Spur documentation?