> For the complete documentation index, see [llms.txt](https://docs.net2grid.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.net2grid.com/net2grid-energy-ai-cloud-interface-specifications/daily-consumer-intelligence/input/measurements.md). # Measurements ## Measurements This document describes the measurement input specifications required for the Daily Consumer Intelligence tier. It defines: * supported measurement metrics * meter types * file structure and upload rules * validation behavior * daily and historical ingestion processes * measurement quality requirements * S3 upload structure *** ### Meter Types NET2GRID EnergyAI® supports two different categories of measurements: | Meter Type | Description | | --------------- | ------------------------------------------------------------------------------------------- | | Mains Meter | Measurements representing the total household energy consumption or production. | | Secondary Meter | Measurements representing the consumption or production of a specific appliance or circuit. | *** #### Mains Meters Mains meters represent the total household energy flow. The supported mains metrics are: * `CSD` * `CSR` * `GAS` These measurements are used as the primary input for: * disaggregation * insights * recommendations * NET2GRID EnergyAI® analytics services *** #### Secondary Meters Secondary meters measure specific appliances or circuits separately from the total household measurements. These measurements can significantly improve the accuracy of NET2GRID EnergyAI® disaggregation and analytics services, since they provide direct appliance-level ground-truth measurements. Secondary meter measurements are optional. *** **Supported Secondary Meter Appliances** The following appliances are currently supported as secondary meters: | Appliance | Identifier | | ----------------------------- | ------------------------------- | | Air Condition | `air-condition` | | Electric Vehicle | `electric-vehicle` | | Battery Energy Storage System | `battery-energy-storage-system` | | Heat Pump | `heat-pump` | | Photovoltaic Panels | `photovoltaic-panels` | *** **Secondary Meter Measurement Types** Secondary meter measurements can be provided using two different approaches: | Type | Description | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | | Dedicated Metering (`D`) | The appliance is measured through a dedicated meter and its measurements are NOT included in the mains household measurements. | | Submetering (`S`) | The appliance measurements are already included in the mains household measurements. | *** #### Dedicated Metering (`D`) Dedicated metering is used when a separate physical meter exists for the appliance. Examples: * dedicated EV charger meter * dedicated heat pump meter * dedicated photovoltaic inverter meter In this case: * the measurements represent only the appliance * the measurements are independent from the household mains measurements *** #### Submetering (`S`) Submetering is used when the appliance measurements are already part of the household mains measurements. Examples: * appliance-level smart plugs * circuit-level submeters * internally estimated appliance measurements In this case: * the appliance measurements are included in the mains household measurements * the measurements are used as additional ground-truth information to improve NET2GRID EnergyAI® analytics *** ### Supported Metrics The following mains metrics are supported: | Metric | Description | Required | | ------ | --------------------------------------- | -------- | | `CSD` | Electricity consumption measurements | Yes | | `CSR` | Electricity return-to-grid measurements | Optional | | `GAS` | Gas consumption measurements | Optional | *** #### Important Notes * `CSR` measurements are optional when no energy production exists in the household. * `GAS` measurements are required only when gas-related services are in scope. *** ### Supported Granularity The following periods of granularities are supported: * 15 minutes * 60 minutes This means that a valid daily file typically contains: | Granularity | Expected Measurements per Day | | ----------- | ----------------------------- | | 60 minutes | 24 measurements | | 15 minutes | 96 measurements | *** ### Measurement Semantics The uploaded measurements represent: * timestamped delta energy values * expressed per interval Measurement values are expected to represent the consumed or produced energy during the interval starting from the provided timestamp until the next interval timestamp. *** #### Supported Units By default, the expected measurement values are in: ``` Wh ``` Support for `kWh` can optionally be enabled through label-partner-specific configuration. *** ### Timezone Requirements Measurements must represent a complete local installation day. Each uploaded daily file is expected to contain measurements covering: ``` 00:00 → 23:45 ``` in the installation local timezone. The timestamps inside the file are expected to correspond to this local-day coverage. *** ## Data Exchange Measurement files are exchanged through AWS S3. The customer uploads the files into the agreed S3 bucket structure. Once uploaded: * the files are validated * valid measurements are imported * downstream NET2GRID EnergyAI® processes may be triggered *** ### File Format Measurement files must: * use CSV format * use UTF-8 encoding without BOM * use comma (`,`) as separator * use dot (`.`) as decimal separator A header row is allowed but not required. *** #### CSV Structure Each row contains: | Field | Description | | --------- | ------------------------------------ | | Timestamp | UNIX timestamp in milliseconds (UTC) | | Value | Delta energy value | *** #### Example ```csv Timestamp,Value 1714521600000,245 1714522500000,301 1714523400000,276 ``` *** ### Directory Structure ### Mains Meter Measurements #### Daily Measurements **Path Format** ``` s3://{bucket_name}/{labelpartner}/measurements/{external-installation-id}/{external-meter-id}/ ``` **Filename Format** ``` {external-installation-id}_{date}_{date}_{metric}.csv ``` *** #### Historical Measurements **Path Format** ``` s3://{bucket_name}/{labelpartner}/historical-measurements/{external-installation-id}/{external-meter-id}/ ``` **Filename Format** ``` {external-installation-id}_{start-date}_{stop-date}_{metric}.csv ``` *** ### Secondary Meter Measurements #### Secondary Meter Metrics Secondary meter measurements use interval-based metric identifiers in the following format: ``` XXXX-PTnHnMnS ``` Where: | Part | Description | | ---------- | ------------------------------------------------------- | | `XXXX` | The metric type identifier | | `PTnHnMnS` | The measurement granularity in ISO-8601 duration format | *** **Metric Type Prefixes** | Metric Prefix | Description | | ------------- | ------------------------------------ | | `0000` | Electricity consumption measurements | | `0001` | Electricity production measurements | *** **Granularity Format (`PTnHnMnS`)** The `PTnHnMnS` section defines the interval between consecutive measurements. The format follows the ISO-8601 duration specification: | Component | Meaning | | --------- | ------- | | `H` | Hours | | `M` | Minutes | | `S` | Seconds | Examples: | Granularity | Metric Suffix | | ----------- | ------------- | | 60 minutes | `PT1H` | | 30 minutes | `PT30M` | | 15 minutes | `PT15M` | *** **Example Secondary Meter Metric IDs** | Metric ID | Description | | ------------ | ----------------------------------------------------- | | `0000-PT15M` | Electricity consumption measurements every 15 minutes | | `0000-PT1H` | Electricity consumption measurements every 60 minutes | | `0001-PT15M` | Electricity production measurements every 15 minutes | *** **Supported Granularity** Secondary meter measurements support: * minimum granularity: 15 minutes * maximum granularity: 60 minutes Only fixed interval measurements are supported. *** #### Daily Measurements **Path Format** ``` s3://{bucket_name}/{labelpartner}/measurements/{external-installation-id}/{external-meter-id}/ ``` **Filename Format** ``` {external-installation-id}_{date}_{date}_{metric-id}_{appliance}_{source}_{unit}.csv ``` *** #### Historical Measurements **Path Format** ``` s3://{bucket_name}/{labelpartner}/historical-measurements/{external-installation-id}/{external-meter-id}/ ``` **Filename Format** ``` {external-installation-id}_{start-date}_{stop-date}_{metric-id}_{appliance}_{source}_{unit}.csv ``` *** ### Path and Filename Parameters | Parameter | Description | | -------------------------- | --------------------------------------------------------------------------- | | `bucket_name` | The AWS S3 bucket agreed between CUSTOMER and NET2GRID. | | `labelpartner` | The label partner identifier provided by NET2GRID during onboarding. | | `external-installation-id` | Unique identifier of the installation/household. | | `external-meter-id` | Unique identifier of the meter providing the measurements. | | `date` | Measurement date in `YYYYMMDD` format. | | `start-date` | Start date of the measurements contained in the file, in `YYYYMMDD` format. | | `stop-date` | End date of the measurements contained in the file, in `YYYYMMDD` format. | | `metric` | Mains metric identifier (`CSD`, `CSR`, `GAS`). | | `metric-id` | Secondary meter metric identifier in `XXXX-PTnHnMnS` format. | | `appliance` | Appliance identifier associated with the secondary meter measurements. | | `source` | Measurement source type. `D` for Dedicated metering. `S` for Submetering. | | `unit` | Measurement unit. | *** ## Measurements Processing *** ### Daily Measurements Process The daily measurements process is the standard ingestion mechanism used after an installation becomes active. *** #### Daily File Rules Each uploaded daily file: * must correspond to a single installation * must correspond to a single metric * must correspond to a single day * must not contain measurements from multiple days * must not contain measurements from multiple installations A separate file is expected for: * each installation * each metric * each day These rules apply to both: * mains measurements * secondary meter measurements *** #### Daily Validation Process Every uploaded daily file passes through a validation process before being accepted for downstream processing. The validation process evaluates: * timestamp consistency * timestamp validity * measurement completeness * invalid or negative values * overall measurement quality The validation is intended to determine whether the uploaded measurements are suitable for disaggregation and downstream analytics. *** #### Minimum Valid Measurements Threshold For a file to be considered valid: * at least 75% of the measurements must pass validation checks If the threshold is satisfied: * the measurements are imported * disaggregation is executed normally If the threshold is not satisfied: * the measurements are considered invalid * disaggregation is not executed * downstream reports may contain invalid or missing results *** ### Historical Measurements Process The historical measurements process is intended for the initial onboarding of a new installation. Historical measurements should be uploaded only once per installation. *** #### Historical Upload Purpose The historical upload process allows NET2GRID EnergyAI® to: * train the installation models * establish historical energy behavior * generate historical analytics * produce historical Disaggregation Categories reports *** #### Historical File Rules Unlike daily uploads: * historical files may contain measurements covering multiple days * historical files may contain large continuous periods of measurements However: * historical uploads are expected to happen only once per installation After the historical upload process has completed: * all future uploads for the installation must follow the daily upload process and use the `measurements` folder * even delayed or backfilled measurements must be uploaded through the daily measurements process These rules apply to both: * mains measurements * secondary meter measurements *** ### Processing Behavior Measurement files are processed asynchronously after upload. #### Measurements Overwrite Behavior NET2GRID EnergyAI® imports measurements using a last-write-wins approach. If multiple uploaded files contain measurements for the same installation, metric, and timestamps: * the latest uploaded measurements overwrite previously imported values This behavior applies to both: * mains measurements * secondary meter measurements This mechanism allows: * correction of previously uploaded measurements * re-upload of improved-quality data * backfilling of historical periods *** ### Measurement Quality Expectations The uploaded measurements are expected to: * contain positive values * contain valid timestamps * follow the expected granularity consistently * cover the full expected daily period * contain sufficiently complete data Poor-quality measurements may lead to: * invalid disaggregation * reduced analytics quality * missing downstream reports *** ### File Processing Reporting File processing reporting is optional and disabled by default. It can be enabled upon request by the label partner. When enabled, NET2GRID publishes processing status messages to an SQS queue after each file is processed. *** #### Processing Report Message Format | Field | Description | | ------------------- | ----------------------------- | | `filename` | Processed filename | | `timestamp` | Processing timestamp | | `error_code` | Processing result code | | `error_description` | Additional processing details | *** #### Processing Error Codes **Successful Processing** | Code | Description | | ----- | --------------------------- | | `000` | File processed successfully | | `010` | File processed partially | *** **Failed Processing** | Code | Description | | ----- | --------------------------------- | | `300` | Installation or account not found | | `400` | Validation failure | | `500` | Internal platform error | *** ### Permissions NET2GRID creates the S3 bucket and grants the customer permissions to: * list objects * upload objects * retrieve objects * delete objects NET2GRID also securely shares the required IAM credentials during onboarding. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.net2grid.com/net2grid-energy-ai-cloud-interface-specifications/daily-consumer-intelligence/input/measurements.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.