Key takeaways
- Keep the raw data unchanged and work from controlled copies.
- Create a data dictionary before interpreting variables.
- Document every cleaning decision and transformation.
- Assess analytical readiness against the research question and design.
Why does data preparation matter?
Analysis can only be as dependable as the data and decisions that support it. Inconsistent coding, undocumented exclusions, duplicate records and misunderstood missing values can change results and weaken interpretation. Data preparation is therefore part of the analytical method, not an administrative task completed before the real work begins.
A transparent preparation process also protects reproducibility. Another informed analyst should be able to understand how the raw data became the analysis-ready dataset and why each important decision was made.
How should raw research data be protected?
Retain an unchanged copy of the original dataset and restrict editing rights. Create a separate working copy for cleaning and analysis. Use clear file names, version dates and a secure storage arrangement appropriate to the sensitivity of the information and the approved research protocol.
Where personal, clinical, institutional or otherwise sensitive data is involved, data access and transfer should follow ethics approval, consent conditions, institutional requirements and applicable privacy obligations. Sensitive datasets should not be attached to a general website enquiry.
| Step | Essential action | Evidence to retain |
|---|---|---|
| Preserve | Protect the unchanged raw data | Read-only original and access record |
| Define | Document variables, codes and units | Data dictionary |
| Check | Review completeness, duplicates and validity | Quality-control log |
| Resolve | Investigate and document corrections | Cleaning decisions or script |
| Assess | Align data with questions and methods | Analysis-readiness summary |
| Version | Save a controlled analysis-ready dataset | Dated file and reproducible workflow |
What belongs in a data dictionary?
A data dictionary explains what each variable represents. It should include the variable name, clear label, data type, unit, permitted values, coding rules, missing-value codes and any derived calculations. For categorical variables, every code should be defined.
This document prevents analysts from guessing. It also reveals problems early, such as two variables that appear to measure the same concept differently or a value that has been used both as a genuine observation and as a missing-data code.
Which data-quality checks should be completed?
Begin with structure and completeness. Confirm that rows and columns represent what you expect, identifiers are appropriate and variable types are correct. Then check duplicates, impossible values, inconsistent categories, unexpected ranges and relationships between fields.
Potential errors should be investigated rather than automatically deleted. An unusual value may be a data-entry problem, but it may also be a valid and important observation. The decision and supporting reason should be recorded.
- Duplicate participant or sample identifiers
- Dates outside the study period
- Values outside possible or protocol-defined ranges
- Inconsistent spelling and category codes
- Units that differ across sites or instruments
- Logical conflicts between related variables
- Patterns and reasons for missing data
How should missing values be handled?
First distinguish a true missing value from not applicable, not measured, below detection, refused or lost. These situations can have different analytical meanings. Summarise missingness by variable and, where relevant, by group, site or collection period.
Do not choose deletion or imputation simply because it is convenient. The appropriate approach depends on the study design, amount and pattern of missingness, planned analysis and assumptions that can reasonably be defended. Important decisions should be discussed with the research team and documented.
When is a dataset ready for analysis?
A dataset is analytically ready when its structure, variables, codes and quality are sufficiently understood; key issues have been resolved or documented; and the available information can address the research questions using appropriate methods. Readiness is not the same as perfection. Remaining limitations should be visible and considered during interpretation.
A Research Data Diagnostic can be useful when the dataset exists but the analytical route is unclear. It can assess structure, quality, alignment with the objectives and the preparation required before a full analysis begins.
Expert support
Would a professional analytical perspective help?
Tell BlackHer Consultancy what you need your data to help you understand or deliver. We will recommend an appropriate next step without asking you to upload sensitive data.
Frequently asked questions
Should I clean my only copy of the dataset?
No. Preserve the raw dataset unchanged and perform cleaning on a controlled working copy so that every change can be reviewed or reproduced.
Can outliers simply be removed?
No. Outliers should be investigated. Removal requires a defensible reason linked to data quality, protocol or analytical assumptions, and the decision should be documented.
What is the difference between data cleaning and data analysis?
Cleaning identifies and addresses quality, structure and consistency issues. Analysis applies appropriate methods to answer the research questions. The two stages are connected, but they have different purposes.
Can BlackHer receive my dataset through the enquiry form?
No. Use the enquiry form to describe the project without attaching sensitive data. Secure transfer and confidentiality arrangements are agreed after the initial scope discussion.