Panel Fraud Index™

Find out which panel providers are delivering high quality survey participants.

How To Use The Panel Fraud Index™

The higher index score the panel partner receives means the quality of the participants is lower and they are blocked from research participation.

The lower index score means the better participant quality and few are removed from participation.

About The Panel Fraud Index™

The Socratic Panel Fraud Index™ is a rating system where Socratic’s panel providers are scored on the level of good and poor participants.
wdt_ID Company Index Score Trend
1 82
2 54
4 57
5 79
6 48
7 35
8 30
9 27
10 24
11 22
12 22
13 21
14 21
15 20
16 20
17 20
18 20
19 19
20 19
21 19
31 19
33 18
34 17
35 17
36 16
37 14
38 14
39 11

Frequently Asked Questions

What Is The Fraud Risk Score™?

An objective measure of the quality of an individual survey participant.

What Is The Panel Fraud Index™?

An aggregated measure of the overall quality of panelists that a panel company delivers to surveys prior to cleaning out the bad actors.

Why was Fraud Risk Score™ (FRS) Created?

The most consistent complaint since the onset of online Marketing Research has been sample quality. It has been an ongoing struggle to keep up with lazy or outright fraudulent survey respondents.

One of the core challenges is that there is not financial incentive for the panel providers to weed out poor quality respondents. Every person they send through that completes and isn’t removed for quality reasons is of value to them. There is no incentive for them to not send that person through for as many times as they can get away with.

How is Fraud Risk Score™ (FRS) Calculated?

As each survey respondent enters our survey platform we perform device fingerprinting tracking using over 300 data points about the participant’s device. We also evaluate IP Reputation using real-time lookups. This allows us to instantly determine the reputation and risk of a potential of a respondent. Finally, when available we look at email reputation with a review of the syntax and DNS to assess the email validity.

The outcome is an individual FRS for each participant. Based on this score can flag them for further review or auto-terminate their survey activity.

The overall quality score of each panel company will be given based on these individuals. This will provide a performance scale for all of our panel partners on the quality of their sample they provide to Socratic.

How is Panel Fraud Index™ (PFI) Calculated?

The PFI is simply an aggregated score of the FRS for each panelist that is sent from a panel company to complete our surveys. Our system stops the bad actors from completing the surveys, but their FSR is included in the PFI calculation.

How Does Socratic Apply FSR and the PFI to their research efforts?

Socratic utilizes the FSR to verify the quality of each individual attempting to complete our surveys. Those with a high FSR are automatically removed or flagged for further manual review.

Socratic utilizes the PFI to monitor the quality of sample being provided from each individual panel company. We work with panel companies to improve the quality of respondents they deliver.

How does FRS benefit Socratic clients?

Socratic strives to provide the best insights to their clients. Ensuring the participants in the research projects who provide information are actually who they say they are and provide relevant and meaningful responses is a critical part of research operations.

The FRS by Socratic gives clients an additional layer of diligence so the insights from the research have a higher quality and trust.

How Often Is the Fraud Risk Score™ and the Panel Fraud Index™ Updated?

The FRS is assigned to every respondent who attempts to complete a Socratic sponsored survey.Therefore in turn, the PFI and subsequent panel rankings is updated with fresh data weekly.

What Happens If You Used A Panel Provider With A High Risk Score For My Research?

Socratic utilizes many panel providers as partners. We often use a mix of different panel companies on each project to avoid single-source bias and ensure we obtain enough quality completed surveys. Because of our different detection approaches, the inclusion of a panel provider with a high PFI means we utilize a smaller number of respondents from that panel (only individuals with a low FRS).

Do you stop working with panel providers with a high Fraud Risk Score™?

If a panel partner has an increase in their PFI we may temporarily suspend using them and discuss their screening procedures to ensure their sample quality improves. We then assign them to our provide sample to our Omnibus surveys and re-engage them into our client surveys once their PFI increases to an acceptable level.

Why don't I see certain Panel providers with a Panel Fraud Index™?

Socratic’s Panel Fraud Index is based on the research activity completed by Socratic with the different panel partners. If a panel provider is not listed with a PFI, it is because either they have not participated in any research with Socratic or have had a limited amount of sample given to Socratic at this time where a Fraud Risk Score can be calculated.