GA4 measurement protocol : missing ability to add traffic source and overwrite IP and user agent [238769739]

Bug

Status Update

No update yet.

Description

bo...@tracedock.com

created issue #1

Jul 13, 2022 10:54AM

__Summary__:
In the classic UA measurement protocol you were able to add traffic source data, and overwrite the user IP and user agent to page_view events.

In the new measurement protocol the description of sending over a page_view with these parameters are missing. Also, adding them manually does not seem to work (both in the body and as URL parameters). Could you provide documentation or clarrification on when this is expected to work?

The fields questioned (as documented in UA measurement protocol):
- _uip_: overwriting IP address
- _ua_: overwriting user agent
- _dr_: location referrer
- _cs, cm, cn, cc, ck, ci_: UTM parameters for source, medium, campaign, term, keyword, campaign id and other advertising identifiers.

__Expected Behavior__:
I would expect the measurement protocol [Reference > events](

https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference/events) to contain a __page_view__ event detailing how to add these fields. As an example, I would expect the fields to be added in the _JSON_ body of the event:

```javascript
const measurementId = `G-XXXXXXXXXX`;
const apiSecret = `<secret_value>`;

fetch(`

https://www.google-analytics.com/mp/collect?measurement_id=${measurementId}&api_secret=${apiSecret}`, {
method: "POST",
body: JSON.stringify({
"client_id": "client_id",
"events": [{
"name": "page_view",
"params": {
"engagement_time_msec": "1501",
"session_id": "123",

"ip": "185.19.103.0",
"user_agent": "Mozilla/5.0 (Android 12; Mobile; rv:102.0) Gecko/102.0 Firefox/102.0",

"page_host": "

https://example.com",
"page_title": "Home page",
"page_referrer": "

https://blog.com",

"utm_source": "

blog.com",
"utm_medium": "blog",
"utm_campaign": "summer",
"utm_term": "referal",
}]
})
});
```

I would expect the following to show up in Google Analytics dashboards, both the real-time, tech overview and user acquisition reports:
- Location of the user based on the IP address overwrite
- Breakdown of the browser technology
- Acquisition of the traffic source based on the UTM tagging

__Steps to Reproduce__:

1. Setup a new empty property
2. Trigger the test code towards this property
3. In _real time_ overview the event shows up, but without a) user location, b) device shows up as desktop and c) no traffic information of the user known
4. User does not show up in later reports around tech, acquisition and engagement.

empty dashboard.png

788 KB

View

Download

Comments

ke...@google.com <ke...@google.com> #2Jul 14, 2022 08:00AM

Reassigned to ke...@google.com.

Hi Boris,

I'm Kevin from the Google Analytics API Team.

Currently, the measurement protocol doesn't support passing IP parameter. Also, our documentation states that "While it is possible to send events to Google Analytics solely with measurement protocol, only partial reporting may be available. The purpose of measurement protocol is to augment existing events collected via gtag, GTM, or Firebase.". You may try implementing custom dimensions to track unsupported values in measurement protocol.

Regards,

Kevin Soriano, Google Analytics API Team

bo...@tracedock.com <bo...@tracedock.com> #3Jul 14, 2022 08:03AM

Hi Kevin,
Thanks for your quick answer, three followup questions for you:

1. Clear in terms of IP address. Is this something that you are still working on, considering the measurement protocol is still in beta?

2. The documentation lacks explanation as to how to send through the user agent? Which field can we use for this, e.g. _ua or ua?

3. The documentation lacks explanation as to how to send page_view events with traffic source fields (e.g. referrer, identifiers, utm parameters) so that it properly categorizes the session and user's source/medium.

cheers

Boris

Message last modified on Jul 14, 2022 12:39PM

ke...@google.com <ke...@google.com> #4Jul 15, 2022 05:47AM

Hi Boris,

Please see below for my comments on your concerns.

Clear in terms of IP address. Is this something that you are still working on, considering the measurement protocol is still in beta?

There's already an existing feature request in order for customers to send IP addresses via Measurement Protocol. However, I'm afraid that we won't be able to provide a timeline on when it will be supported. Can you provide your use-case to me on why you need to send IP addresses via MP so I can add it to the ongoing feature request?

How can I send through the user agent?

Can you provide your use-case to me on why you need to send user agent via MP so I can share it to our team as well?

How can I send page_view events with traffic source fields (e.g. referrer, identifiers, utm parameters) so that it properly categorizes the session and user's source/medium.

I'll check this with the rest of our team. I'll let you know once I have more information. In the meantime, you can try creating custom dimensions to track these information.

Regards,

Kevin Soriano, Google Analytics API team

bo...@tracedock.com <bo...@tracedock.com> #5Jul 19, 2022 07:03AM

Hi Kevin,

Did you reach out to the team to discuss question 3? I'd like to add traffic source fields to pageviews in order to properly attribute the user & session to the proper channel grouping. I've tried to add the fields as acronym (e.g. dr and cm) to the URL GET parameters in the request, as well as writing them in full in the JSON body (e.g. page_referrer and campaign_medium). None of the options result in attributing the user to the proper channel grouping in the attribution report (after waiting 24 hours for GA4 to process them).

In terms of usecase for the user agent: For devices without GA4 support we'd like to forward anonymous data for attribution and reporting purposes. Does that make sense?

Let me know if the team has answered you,

thanks in advanced,

Boris

ke...@google.com <ke...@google.com> #6Jul 20, 2022 04:59AM

Hi Boris,

Yes, I've reached out to our team. However, I'm afraid that I don't have any updates yet. Have you tried to create custom dimensions and generate reports based on those dimensions? How did it go?

Also, just to clarify before I raise a feature request on user agent, are you associating your MP events with client_id/app_instance_id from gtag/SDK? Or are you attempting a pure server-to-server implementation?

Regards,

Kevin Soriano, Google Analytics API team

bo...@tracedock.com <bo...@tracedock.com> #7Jul 30, 2022 09:19AM

Hi Kevin,

Did you reach out to the team and get an answer as to how send over traffic source fields?

User agent, the use case is that we have devices without GA4, and we need to send over server based hits. We'd like to add information that the device is mobile or tablet for example.

cheers

Boris

ke...@google.com <ke...@google.com> #8Aug 1, 2022 07:45AM

Hi Boris,

Regarding traffic source fields, it's not yet supported via Measurement Protocol. We have raised a feature request to add support for traffic source fields as well as the user agent. Please note that we are unable to provide a timeline on when these fields will be supported. You may follow Google Analytics API Notify group for future updates.

Regards,

Kevin Soriano, Google Analytics API Notify group

b....@gmail.com <b....@gmail.com> #9Sep 6, 2022 11:22AM

Hi Kevin,

We are struggling with implementing the measurement protocol as long as traffic source fields for pageviews are not yet supported. Have you any news on the team? I know you can't communicate timelines, but perhaps you can indicate if this bug you raised with the will be picked up.

Many thanks,

cheers

Boris

ke...@google.com <ke...@google.com> #10Sep 7, 2022 07:07AM

Hi Boris,

I'm afraid that I don't have any updates on this. I'll share your latest feedback with the rest of our team. However, I can only suggest that you follow the Google Analytics API Notify group for future updates.

Regards,

Kevin Soriano, Google Analytics API team

b....@gmail.com <b....@gmail.com> #11Sep 7, 2022 07:55AM

Hi Kevin,

Clear that you are not able to share timelines.

But do you know if the team is planning to pick this up in the future, or have they decided not to implement these updates?

cheers

Boris

ke...@google.com <ke...@google.com> #12Sep 8, 2022 02:02AM

Hi Boris,

Our team will evaluate the feature request, but I'm afraid that I can't comment if it will indeed be implemented in the future or not.

Regards,

Kevin Soriano, Google Analytics API Team

je...@yoursurprise.com <je...@yoursurprise.com> #13Sep 12, 2022 10:21AM

Hi Kevin,

Chiming in here since we're also running into an issue with the user agent and/or device type when using the measurement API.

We've got an e-commerce website where we're tracking the majority of our events through the measurement API as implemented by Segment.
Currently only the pageviews are being measured client-side through gtag.js, and we use the resulting session_id, session_number, and client_id to stitch the events sent sent through the measurement API to user's sessions.

It seems like all of these events are being flagged with a (not set) browser, which is understandable if the API doesn't support setting the user agent or browser.
However, GA4 seems to then think that an unknown browser is automatically a desktop device, which is a bit misleading.

This leads to the following discrepancies where 70% of our new users are on mobile (since those will be measured through gtag.js), but 60% of all users are on desktop. (see attached screenshots)
It's hard to gain good insights with discrepancies such as these.

Wouldn't it be better to have a device type (not set) as well when the user agent is (not set)? Or to just exclude these data points from device type reporting?

Regards,
Jeroen

new user devices.png

60 KB

View

Download

all user devices.png

63 KB

View

Download

ke...@google.com <ke...@google.com> #14Sep 13, 2022 06:54AM

Hi Jeroen,

Thank you for providing more information in this thread. I've shared it with the rest of our team as a feature request. You may follow the Google Analytics API Notify group for future updates.

Regards,

Kevin Soriano, Google Analytics API team

yu...@gmail.com <yu...@gmail.com> #15Sep 15, 2022 12:35PM

Hi，Kevin，
The same requirement, I am a website developer and manager, can reduce the user's resource request when requesting from the server (

googletagmanager.com and

google-analytics.com cannot be requested normally in many places); the solution is to directly on the server side Request (with Google Analytics 3 the cip parameter is used, but UA4 does not work); please increase this parameter.

Regards,

ke...@google.com <ke...@google.com> #16Sep 16, 2022 07:11AM

Hello,

Can you confirm if you are referring to uip so I can let our team know that you are looking for this feature as well? I'm asking as I'm unable to find a cip parameter in the parameter reference of UA.

Regards,

Kevin Soriano, Google Analytics API Team

yu...@gmail.com <yu...@gmail.com> #17Sep 16, 2022 08:42AM

😱，it's uip !!!

<buganizer-system@google.com>于2022年9月16日周五下午3:11写道：

- Show quoted text -

ke...@google.com <ke...@google.com> #18Sep 19, 2022 06:15AM

Hi,

Thank you for the clarification.

Have you tried to check if custom dimensions would satisfy your needs? If not, please provide more details so I can share it with our team.

Regards,

Kevin Soriano, Google Analytics API team

bl...@converteo.com <bl...@converteo.com> #19Sep 26, 2022 09:06AM

Hello Kevin,

Any update on this issue ?
Being able to manipulate IP adress, user agent, referrer and campaign parameters, is not only necessary to implement a pure server side tracking for GA4, but also to comply with EU regulations for the usage of GA (and although google provide some privacy controls in GA4 to redact some parameters, it doesn't cover all the requirements for some countries, and doesn't let us categorize the traffic ourselves).
And while custom dimensions can be used as a stopgap, it won't let us use the full features of GA4 (like attribution etc.).

Best regards,

ke...@google.com <ke...@google.com> #20Sep 27, 2022 06:42AM

Hello,

I'm afraid that I don't have any updates that I can share regarding the feature requests. Are you able to further elaborate on your use case so I can share it with the rest of our team? This would help our team to evaluate the feature.

Regards,

Kevin Soriano, Google Analytics API team

bl...@converteo.com <bl...@converteo.com> #21Sep 28, 2022 09:01AM

Hello Kevin, thanks to your reply.

In our case the CNIL (french internet privacy regulatory agency) outlined the only way for Google Analytics to be compliant for GDPR was through a server-side implementation that redacts the IP, query parameters (incl. UTMs and internal search keywords), referrer, as well as any cross-site IDs. In addition we have to hash the client IDs and truncate the user_agent.
A number of these settings are already available in GTM server-side with the GA4 tags (in the resulting http request we can see parameters for _uip for the ip adress, or user_agent for the user agent ) with the option to redact or modify some of these parameters but there is no documentation on these for the GA4 measurement protocol.

Simply losing all these parameters would make the usage of GA4 quite limited but if we are able to manipulate this data through measurement protocol, we could reprocess the data and send it in a way that is compliant with the regulations and limit the loss of data. For example send generic medium/sources for campaigns instead of granular data from the URL and referrer)

Best Regards,

ke...@google.com <ke...@google.com> #22Sep 30, 2022 06:15AM

Hello,

Thank you for providing more information. I'll be sharing these with our team. You may follow the Google Analytics API Notify group for future updates.

Regards,

Kevin Soriano, Google Analytics API team

is...@google.com <is...@google.com> Oct 8, 2023 08:06AM

Status: New

Issue 238769739

Description

Issue summary

Comments

ke...@google.com <ke...@google.com> #2Jul 14, 2022 08:00AM

bo...@tracedock.com <bo...@tracedock.com> #3Jul 14, 2022 08:03AM

ke...@google.com <ke...@google.com> #4Jul 15, 2022 05:47AM

bo...@tracedock.com <bo...@tracedock.com> #5Jul 19, 2022 07:03AM

ke...@google.com <ke...@google.com> #6Jul 20, 2022 04:59AM

bo...@tracedock.com <bo...@tracedock.com> #7Jul 30, 2022 09:19AM

ke...@google.com <ke...@google.com> #8Aug 1, 2022 07:45AM

b....@gmail.com <b....@gmail.com> #9Sep 6, 2022 11:22AM

ke...@google.com <ke...@google.com> #10Sep 7, 2022 07:07AM

b....@gmail.com <b....@gmail.com> #11Sep 7, 2022 07:55AM

ke...@google.com <ke...@google.com> #12Sep 8, 2022 02:02AM

je...@yoursurprise.com <je...@yoursurprise.com> #13Sep 12, 2022 10:21AM

ke...@google.com <ke...@google.com> #14Sep 13, 2022 06:54AM

yu...@gmail.com <yu...@gmail.com> #15Sep 15, 2022 12:35PM

ke...@google.com <ke...@google.com> #16Sep 16, 2022 07:11AM

yu...@gmail.com <yu...@gmail.com> #17Sep 16, 2022 08:42AM

ke...@google.com <ke...@google.com> #18Sep 19, 2022 06:15AM

bl...@converteo.com <bl...@converteo.com> #19Sep 26, 2022 09:06AM

ke...@google.com <ke...@google.com> #20Sep 27, 2022 06:42AM

bl...@converteo.com <bl...@converteo.com> #21Sep 28, 2022 09:01AM

ke...@google.com <ke...@google.com> #22Sep 30, 2022 06:15AM

is...@google.com <is...@google.com> Oct 8, 2023 08:06AM

Add comment

Issue metadata