日向坂46メッセージ取得用の refresh_token が取得できない

[kk-shinoda]

概要

refresh_token の取得手順 の mitmproxy で解析中に https://api.kh.glastonr.net/v2/signin へのリクエストが見つからないため refresh_token が取得できない

情報

関係ないものもあるかもしれませんが、調査に関する情報を記載します。

• 実行環境
  • PC の OS: macOS Sequoia (intel)
  • スマートフォンの OS: iOS 18.0
  • 日向坂46メッセージのバージョン: 3.1.6
• 以前取得した refresh_token を用いて colmsg を実行すると問題なくデータ取得ができる
• 2024年8月に日向坂46メッセージアプリが大型アップデートしており、その影響を受けている可能性がある
• mitmproxy のログに TLS のバージョンによるエラーが発生している

  Client TLS handshake failed. Client and mitmproxy cannot agree on a TLS version to use. You may need to adjust mitmproxy's tls_version_client_min option.
  

[axlchr12]

I'm facing the same issue.

I think it only captures the /update_token only, not /signin.

[payt0nc]

The app looks like enabled SSL pinning. The update should be a month before with UI amendment.

This feature would only trust the application stated SSL Certification rather than the MITM certification.

What if you are using iOS, it's very difficult to capture. But, Android is still possible.

Also, another stuff is the refresh_token would be updated when you switch login between device. It means when you get the refresh_token from an Android device, but you often use your iPhone for reading messages as usual, the refresh_token would be change at the moment of iPhone login.

This is my observation.

[proshunsuke]

Thank you for issue and investigating.

It seems that getting the refresh_token has become difficult on iOS.

Even on Android, it is difficult to use mitmproxy on devices running version 7 or later, making it hard to retrieve the refresh_token. Moreover, as mentioned in #108 (comment) , even if you set up an environment with Android 6 or earlier and use mitmproxy to get the refresh_token, it would be inconvenient since the refresh_token would be used in a different environment from your usual device.

Currently, it is difficult for new users to use colmsg on both iOS and Android (though existing colmsg users who have already got the refresh_token will not be affected).

Unfortunately, I don't have a complete solution for this issue at the moment.

For now, I will add a notice about this issue at the top of the repository.

[axlchr12]

I think, for now, the best approach is to avoid updating or downgrading the app if you're using Android.

I've tested on both my iOS and Android phones and successfully retrieved the refresh_token via MITM on older app versions (below version 3.0.0).

iOS
櫻坂46メッセージ & 乃木坂46メッセージ: Version 3.1.6.33
日向坂46メッセージ: Version 2.16.01.224 (still works)

Android
All versions are still working
櫻坂46メッセージ: Version 1.13.00.180
乃木坂46メッセージ: Version 1.11.01.224
日向坂46メッセージ: Version 2.14.00.180

So, yeah, they enabled SSL pinning after the major UI update in August 2024. Luckily, we can still use the older app versions.

[kk-shinoda]

I have confirmed that the refresh_token can be obtained by using an emulator running in Genymotion on Windows and Intel-based Macs.

In Genymotion, start a device with Android 6.0 or below, and download the "日向坂46 メッセージ" app version below 3 from APKPure.

After that, you can obtain the refresh_token using mitmproxy as usual.

I haven't figured out a way to run an Android 6.0 or earlier emulator on an Apple Silicon Mac, so the same method doesn't work. If you have any ideas, I'd appreciate your help.

[yoshyv]

Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages

[payt0nc]

Force updated has been released. Since 2400-2800, Dec.18, 2024, both iOS and Android devices would be requested to use latest version.

So, from Dec.19, 2024, it is highly impossible to retrieve refresh_token by using mitm.

[axlchr12]

Yup, I got the notification too. I guess this is the end for us?

[payt0nc]

Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages

Yes, but if you have already retrieved the access_token.

According the flow of application,

refresh_token is using for updating the latest access_token in for get data. (Typical OAuth2.0 application flow)
So, when you have owned refresh_token and well protected this refresh_token, it would let you get access_token.

But, the things we discussed above are related to SSL pinning. In other words, the case all communications are protected with the SSL certification. It's very difficult to capture the communication between the application and server.

[payt0nc]

Yup, I got the notification too. I guess this is the end for us?

Maybe. It's becoming those good old days

Cheers mates! 🍻

Thanks @proshunsuke to contribute this project.
本当にありがとうございます！

[axlchr12]

Can we use access_token to download as many as possible as a temporary solution? access_token expires after an hour iirc which is enough to download quite a lot of messages

Yes, but if you have already retrieved the access_token.

According the flow of application,

refresh_token is using for updating the latest access_token in for get data. (Typical OAuth2.0 application flow) So, when you have owned refresh_token and well protected this refresh_token, it would let you get access_token.

But, the things we discussed above are related to SSL pinning. In other words, the case all communications are protected with the SSL certification. It's very difficult to capture the communication between the application and server.

So, it means that despite having an access token, we still can't do anything because of SSL pinning?

[axlchr12]

Yup, I got the notification too. I guess this is the end for us?

Maybe. It's becoming those good old days

Cheers mates! 🍻

Thanks @proshunsuke to contribute this project. 本当にありがとうございます！

It's kinda sad, though. I really need this for when a member graduates or for monthly backups. I guess this might be the end for us—or maybe not. I hope the Chinese Weibo fans can do something about it.

[payt0nc]

So, it means that despite having an access token, we still can't do anything because of SSL pinning?

Because access_token would be expired, but refresh_token seems only updated when application execute transfer or login flow.

So, once you have refresh_token, it just use is to get access_token from /login endpoint.
Without SSL pinning, we could easily capture either refresh_token or access_token. Under SSL pinning, we can not see this communication from mitm

[axlchr12]

So, it means that despite having an access token, we still can't do anything because of SSL pinning?

Because access_token would be expired, but refresh_token seems only updated when application execute transfer or login flow.

So, once you have refresh_token, it just use is to get access_token from /login endpoint. Without SSL pinning, we could easily capture either refresh_token or access_token. Under SSL pinning, we can not see this communication from mitm

Yeah, I mean, as long as we have the refresh_token and don't do anything like transfer or login flow, we can still do it, right?

[payt0nc]

Yeah, I mean, as long as we have the refresh_token and don't do anything like transfer or login flow, we can still do it, right?

Seems like that.

[proshunsuke]

Thank you for notifying about the forced update.

As explained, once a valid refresh_token is retrieved, it can be used to receive messages. However, the token will be updated through the login flow.

Due to the forced update, it will become difficult to retrieve a new refresh_token after Dec. 19. If a valid refresh_token remains after Dec. 19, it will likely still allow messages to be received. However, I’m not sure how future updates will affect this.

[axlchr12]

I have confirmed that the refresh_token can be obtained by using an emulator running in Genymotion on Windows and Intel-based Macs.

In Genymotion, start a device with Android 6.0 or below, and download the "日向坂46 メッセージ" app version below 3 from APKPure.

After that, you can obtain the refresh_token using mitmproxy as usual.

I haven't figured out a way to run an Android 6.0 or earlier emulator on an Apple Silicon Mac, so the same method doesn't work. If you have any ideas, I'd appreciate your help.

I'm trying to use Genymotion on my Linux and I'm encountering an error like this. Do I need to use Android 7.0 or higher?

[payt0nc]

Wait... maybe it's just a technical problem on Flutter...

Flutter applications still don't automatically use the system's proxy, unless the developer adds this functionality by creating custom Android & iOS plugins that provide this information.

https://blog.nviso.eu/2022/08/18/intercept-flutter-traffic-on-ios-and-android-http-https-dio-pinning/

[axlchr12]

Wait... maybe it's just a technical problem on Flutter...

Flutter applications still don't automatically use the system's proxy, unless the developer adds this functionality by creating custom Android & iOS plugins that provide this information.

https://blog.nviso.eu/2022/08/18/intercept-flutter-traffic-on-ios-and-android-http-https-dio-pinning/

Oh? So this Mobame app uses Flutter? I just found out...

I tried using Reqable with the 'Bypass All SSL' feature yesterday, yet it still did not capture the API hits.

[ktsh2024]

I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.

[samuel626]

I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.

can you provide which emulator you are using and maybe the process? I tried with nox and seems not working

[ktsh2024]

I am able to get the token by running the latest APK on an Android Emulator. Do a scan using Cheat Engine on the emulator process. It takes some time to try an error to inspect the memory.

can you provide which emulator you are using and maybe the process? I tried with nox and seems not working

I am using LDPlayer 9. The process name is Ld9BoxHeadless.
At the log in confirmation screen, search for the refresh token from the memory.