The gathering stage involves acquiring the vital data with a purpose to perform a meaningful evaluation based upon accurate information.
Techniques
Data Requirements
Define which data is required to properly approach the project (e.g. format, variables, time range, granularity)
Data Sources
Find reliable and relevant data sources (e.g. databases, APIs, files, sensor readings)
Authentication
Secure vital permissions to access the information (e.g. email/password, OAuth, API key, robots.txt)
Collection
Acquire the information using appropriate methods (e.g. SQL queries, API calls, web scraping, manual data entry)
Data Management
Handle the information in accordance with best practices (e.g. data quality, data governance, data security)