An insurance premium is not a price on a shelf. It is the outcome of a calculation, based on who you are, what you want to cover and under which conditions. Ask two insurers for a quote for the same customer and you get two numbers that cannot simply be set side by side.
That is what makes insurance benchmarking different from comparing products with a barcode. You have to define what "comparable" means before you compare anything.
A premium comes out of a quote flow. The customer enters a profile, such as age, postcode and claims history, chooses a cover and a deductible, and the insurer calculates a price. There is no fixed label to read from a page, and the same product can show many different prices. A benchmark therefore needs a defined set of profiles and cover choices, and a way to collect the resulting premium for each of them.
Two quotes are comparable when the cover behind them is equivalent. That depends on the scope of the cover, the deductible, the sum insured, optional extras, the policy term, and the conditions and exclusions. Leaving one out does not simplify the comparison: it makes it wrong in a way that is hard to notice.
A useful approach is to split attributes into those that must match, such as the type of cover and the deductible, and those that contribute to a weighted score, such as optional extras. A cheaper quote with a higher deductible or a narrower cover is a different proposition. We describe this logic in more detail in Matching without a barcode: How to compare prices when there's no EAN.
Choose profiles that reflect your actual customers. Vary the attributes that drive the premium, such as age, location, claims history and the property or vehicle insured, and document each profile so the measurement can be repeated. Use fictional test profiles, never real personal data. Start with a limited set that covers your largest segments and extend it only when extra profiles answer a commercial question.
Insurers price risk differently. One may be sharp for young drivers and expensive for older ones, while another does the opposite. A benchmark based on a single profile only tells you how you compare for that profile. A set of profiles, weighted by the importance of each segment in your portfolio, shows where you are competitive and where you are not.
Premiums are shaped by more than the underlying risk. Bundle discounts, first-year discounts, extras that look free and surcharges for monthly payment all change what the customer actually pays. Normalise every quote to a common basis, such as the total annual cost including fees and surcharges, before comparing. Otherwise you reward whoever packages the offer most cleverly, not whoever is genuinely more competitive.
Collecting premiums at scale means completing quote flows the way a customer would. That comes with the same challenges as other configurators: the number of combinations is large, flows change, and a small change can break a scraper without anyone noticing. Store each premium together with its profile, cover choices and time of collection, validate every run, and detect changes in the flow early. We cover this in Configurator scraping: how to monitor prices that only appear after configuration.
Monitoring only works for quote flows that are publicly accessible. Flows that require a login or personal identification are usually not suitable. Check the terms of use of the sites involved before you start.
Deciding which profiles and covers matter is something you are well placed to do. What is harder to sustain is the engineering and upkeep: completing quote flows at scale, matching comparable covers, and keeping everything accurate when competitors change their flows or products.
At Competify, we do this for markets such as insurance, automotive leasing, hospitality and telecom. We agree with you which profiles and covers matter, build and maintain the data collection, match and validate the results, and deliver them through a dashboard, API or file transfer. If you would rather not own that part yourself, please plan an introduction meeting.
Want to know how we work?
We build and maintain custom webscrapers that turn any website into structured data and insights.
