Session Details:
Protecting Data from Users: Some Thoughts on Guidelines
Statistical software is so easily available, and at so low a cost, that any software user with data can execute assorted analyses by themselves. There may be little or no intervention or analytic oversight guiding the process. A point of great concern, amplified by omnipresent “data science,” is the ease with which software users can torture data through analyses without fully appreciating or understanding the information extracted. There are consequences, risks, affecting conclusions reached and decisions taken. A more subtle way of addressing this concern, for me, has been to suggest building protections into the process. And just to be sure, by “protections” is meant “protecting data from the user”.
Data are meant to be innocent (themselves having no agenda) pieces of information, evidence, then reduced via analysis to some insight or result upon which a decision can be made regarding “next steps”. At issue then is how can data be analyzed, and reported, protecting this information, ensuring a reasonably clear direction for the required decision, reported with an equally clear sense of risk assessment.
Using Data to set Action Standards for Decision Making an Introduction to Risk Assessment
Every product test is an experiment. Their data collected are used as the basis for business decisions and planning next development steps.
Every statistical test run on the data advises on the direction of those decisions and each test is accompanied by risk of advising poorly.
Statistical test results run the risk of making one of two errors:
(1) making the tester believe there is a difference between the products being tested when there are none.
(2) making the tester believe there is no difference between the products being tested when there is.
The goal here is provide a context for risk:
• To help the tester quantify the risk (in a monetary sense, if possible)
• Adjust the statistical testing process accordingly and reduce, or at least prepare for, the risk.
An Introduction to Planning and Design of Product Tests with an Eye on Controlling Order Effects
Product tests are all experiments of a sort. To plan and execute them well, consideration needs be given to:
· Treatment Structure: the nature of the products to be tested
· Design Structure: who sees what and when
· Analysis of the data collected
Often, these tests ask respondents to evaluate more than one product, creating order effects: the influence of the context within which a product is evaluated. These effects can be “balanced out” with an appropriate design… but the design itself does not nullify the effects. Only with an appropriate design can these effects be measured,...
Session Details:
Protecting Data from Users: Some Thoughts on Guidelines
Statistical software is so easily available, and at so low a cost, that any software user with data can execute assorted analyses by themselves. There may be little or no intervention or analytic oversight guiding the process. A point of great concern, amplified by omnipresent “data science,” is the ease with which software users can torture data through analyses without fully appreciating or understanding the information extracted. There are consequences, risks, affecting conclusions reached and decisions taken. A more subtle way of addressing this concern, for me, has been to suggest building protections into the process. And just to be sure, by “protections” is meant “protecting data from the user”.
Data are meant to be innocent (themselves having no agenda) pieces of information, evidence, then reduced via analysis to some insight or result upon which a decision can be made regarding “next steps”. At issue then is how can data be analyzed, and reported, protecting this information, ensuring a reasonably clear direction for the required decision, reported with an equally clear sense of risk assessment.
Using Data to set Action Standards for Decision Making an Introduction to Risk Assessment
Every product test is an experiment. Their data collected are used as the basis for business decisions and planning next development steps.
Every statistical test run on the data advises on the direction of those decisions and each test is accompanied by risk of advising poorly.
Statistical test results run the risk of making one of two errors:
(1) making the tester believe there is a difference between the products being tested when there are none.
(2) making the tester believe there is no difference between the products being tested when there is.
The goal here is provide a context for risk:
• To help the tester quantify the risk (in a monetary sense, if possible)
• Adjust the statistical testing process accordingly and reduce, or at least prepare for, the risk.
An Introduction to Planning and Design of Product Tests with an Eye on Controlling Order Effects
Product tests are all experiments of a sort. To plan and execute them well, consideration needs be given to:
· Treatment Structure: the nature of the products to be tested
· Design Structure: who sees what and when
· Analysis of the data collected
Often, these tests ask respondents to evaluate more than one product, creating order effects: the influence of the context within which a product is evaluated. These effects can be “balanced out” with an appropriate design… but the design itself does not nullify the effects. Only with an appropriate design can these effects be measured, controlled, and used to better evaluate product performance.
