SOLD! – An auction mechanism for equitable and fair data collection

From pharmaceutical companies harvesting digital medical histories for product development to e-commerce companies wanting to refine their online advertising, firms rely heavily on individual consumer data to inform their decisions. Yet, collecting this data is fraught with a multitude of challenges including biased datasets, regulatory pressures of consent, issues of trust, and fair compensation. 

This is the focus of BTMA faculty members Dr. Hooman Hidaji and Dr. Raymond Patterson along with co-authors^[1] in their recent paper Algorithms to the Rescue: Market Mechanisms for Consensual Trading of Unbiased Individual Data, accepted for publication at Information Systems Research^[2]. Specifically, the paper addresses many of the above issues by developing a novel algorithmic market mechanism for collecting unbiased individual-level data that allows fair participation at near-optimal compensation.

Using a random sampling method, their mechanism pairs users based on correlated privacy and sensitivity attributes and advances an auctioning system to determine which one gets paid and by how much. For example, two users may join a platform collecting survey data and be paired based on similar reported attributes, reducing bias when selecting one of them, and then be compensated via a second-price auction (i.e., the ‘winner’ user is paid the losing bid’s higher price). This pairing and auction process is repeated to complete a desired sample size (e.g., 1,500 users).

“As a consumer, you’ll know that win or lose, it is fair and ethical.”

Patterson states that the mechanism outperforms other payment approaches at the lowest possible cost, 

“The result is ethical payments to data subjects, with data samples that are naturally better than what would be otherwise possible. And it is substantially cheaper than the current flat-rate method. Everybody wins.”

While previous research in this arena focuses on highly complex mathematical solutions to privacy concerns, this research takes a more practical approach. By sampling sensitive attributes (e.g., income, health status, age) matched with a privacy preference (a value representing how much they care about privacy), users are motivated to report their true privacy preference, which yields high data quality. 

Patterson explains,

“Our market mechanism induces you to tell the truth. And it is fair to you. By pairing you with another person who has a similar privacy preference profile, and if you win the bid, you’ll be paid the higher amount that the other person bid as their price. Thus, you’ll have the peace of mind that you’ll be paid at least as much as what you asked. If you lose the competition, then your data is not released. As a consumer, you’ll know that win or lose, it is fair and ethical.”

Results hold under imperfect correlation of pairings showing the real-world potential of the research findings. 

“It works effectively when uncertainty about privacy concerns exceeds a threshold, which holds true for most practical scenarios. Results remain robust under varying correlation levels between privacy concerns and sensitive attributes”, says Hidaji, “What we found surprising was that our approach - without any access to information about data subjects’ privacy concern - generally outperforms approaches with access to partial information”, he continues.

The inspiration for the study grew out of real-world applications identified with Numerous Limited. The former digital solutions company used zero-party data to improve the online consumer shopping experience for e-commerce firms. 

The needs of Numerous Limited informed the study, 

“Our study was shaped based on our conversations with Numerous Limited. Their marketplace and its challenges as a platform were what drove us to look for a better mechanism. We believe that our approach is one solution for a particular type of platform: one which aims to sell samples of data to buyers”, states Hidaji.

Researchers recognize that an evolving regulatory environment which enforces transparency and acquisition of consent for using subjects’ data is becoming more wide-spread, making the mechanism applicable even for scenarios where trust by the user may be unknown, but data is still being collected. 

And while researchers have no immediate plans to implement the mechanism with another company, they are open to the idea. But Hidaji believes that the paper can also stand on its own.

“We would be open to discussing our approach with any company that may be interested. As well, due to the relative simplicity of our approach, platforms may be able to implement the mechanism based on the paper”, he says. 

Dr. Hidaji and Dr. Patterson continue their research in BTMA on similar and other topics including economics of information systems, data privacy and protection regulation, and platforms (Hidaji) and information systems, analytics, and quantitative decision and artificial intelligence technologies (Patterson).

_____________________________________________________________________

^{[1] Co-authors are Brian Birkhead, Ashkan Eshghi, and Ram D. Gopal}

^{[2] Birkhead, B., Eshghi, A., Gopal, R. D., Hidaji, H., & Patterson, R. A. (2025). Algorithms to the Rescue: Market Mechanisms for Consensual Trading of Unbiased Individual Data. Information Systems Research https://doi.org/10.1287/isre.2024.1115}