FullStockWipe and HTC OTA's a.k.a "Verity"
What's the problem?
The way the new security works, a FullStock zip will break your OTA capability in almost all scenarios. The only scenario where that would NOT happen is if you have, before you flash the zip, already the corresponding untouched, hash-sum matching stock system image flashed. Nobody using custom ROM's has that. So, what happens is this:
- FullStock.zip flashes stock kernel which has verity enabled and checks partition integrity upon boot.
- After flashing, you reboot. The kernel kicks in (around when bootanimation would start). The kernel checks the /system partition if it is the correct one.
- The kernel finds it is not the correct system and reports a fail-status, sets this as persistent information, and will force a reboot
- At reboot, (and every reboot after) the Bootloader picks up the fail status and pass it on to the kernel, which in turn will pass it on to the system.
- Now, Android thinks, System is messed up and will not allow you to download and apply an OTA.
From now on, every boot, even if you flash a clean stock system, the aboot will tell the rest of the guys working inside your phone: hey, this thing has been messed with, it cannot be updated anymore!
Fixing strategies:
1.) To restore OTA function fast and easy: run a RUU.
2.) If there is no RUU for your specific model, you could convert to another model which has a RUU.
3.) Then there is a third, theoretical way I have not yet tested: obtain an untouched system image that fits your FullStock (same version), flash that in fastboot (it can be a raw dump or a TWRP systemimage backup of the correct system) and then flash the corresponding FullStock again. That should, like a RUU, restore OTA functionality too.
4.) Also very simple: grab the HTC OTA file which you find in /data/data/com.htc.updater somewhere if you can download. If not, find it on XDA from someone else who got it. Then put it on your sdcard, have stock recovery flash it from SDCard (no detailed guide here but its the same since years, there are tons of guides on how to manually flash a HTC OTA out there. Use Google).
5.) Remove boot.img from my FullStock zip before you flash. Your custom ROM of choice will put a hacked kernel into your phone again anyway...
However, at time of writing this, I know
@nkk71 is investigating other, simpler methods to restore the correct state.
So, if you absolutely depend on HTC's OTA's, best would be to just not flash/boot a stock kernel ever while a custom system is installed.
Flash Process Output (applies mostly to older phones, the newer HOSD driven output is much more detailed)
There are a few steps in the flash process which are not really straightforward but i can maybe explain some of them here:
sending 'zip' means: fastboot is sending zip over to client (here referred to as remote)
OKAY [ 2.839s] means status of sending was good. Transfer succeeded.
writing 'zip'... means the zip is being written to some location on the phone from the /temp location.
(bootloader) zip header checking... means the zip header is being checked for validity, see if it's a real zip file and check for HTC's signature, which often resides in the header part.
(bootloader) zip info parsing... means most likely a check on the file hashes in the zip (integrity check - if the zip is borked, it will fail here)
(bootloader) checking model ID... The bootloader checks if the android-info.txt contains the right MID. If it fails here you gotta swap out your model ID in the android-info.txt file or write another MID to your phone.
(bootloader) checking custom ID... The bootloader checks if the android-info.txt contains the right CID. If it fails here you gotta swap out your Customer ID in the android-info.txt file or write another CID, possibly SuperCID, to your phone.
(bootloader) start image[hboot] unzipping for pre-update check... means the bootloader is now unzipping the [hboot] image. This line will be repeated before every image that is to be flashed.
(bootloader) start image[hboot] flushing... means the bootloader is now beginning to flash the [hboot] image.
(bootloader) [RUU]WP,hboot,0
(bootloader) [RUU]WP,hboot,99
(bootloader) [RUU]WP,hboot,100 these three lines read [RUU] for what mode fastboot is in, WP for Write Partition for what is currently being done in RUUmode, hboot is the name of the currently flashed partition, number xx is a percent stage of the write process.
(bootloader) ...... Successful means the final status is successful.
Now, before the [RUU]WP,hboot,xx line we often see another line reading [RUU]UZ,radio,50 for example. That reads RUUmode is currently unzipping the Radio.img and at stage 50 percent. UZ means UNZIP.
If you see something like this:
(bootloader) start image[sbl1-1] unzipping & flushing...
(bootloader) [RUU]UZ,sbl1-1,0
(bootloader) [RUU]UZ,sbl1-1,100
(bootloader) signature checking... means it is checking the signature of the partition if it matches the expexted signature stored in the hboot.
(bootloader) verified fail means the signature in the image did not meet expectations.
(bootloader) ..... Bypassed means the image got skipped because its got the wrong signature.
This has to be interpreted like this: there are multiple SBL images, to be exact: type 1 has 3 variants and type 2 has only one variant. Of type 1 (SBL1-x), two get skipped, one gets flashed (see my log above), of type two (SBLx) both get flashed. I believe, SBL 2 and 3 are device independent, but SBL1 has three variants, of which only one fits the current device. So, depending on the device you have, you will see either SBL1-1, SBL1-2 or SBL1-3 being flashed and the other two subtypes being skipped (bypassed).
The same goes for the "dzdata" images in the firmware package. They come in two or three size flavors (16, 32 and 64 GB) and resemble the file structure of the /data partition. Depending on your device and model, only the one with the right size gets flashed, the others skipped.
Important to understand: nearly all FAILED messages that do NOT occur while [RUU]WP (write partition) should be considered harmless. Only a FAIL during a write operation will most likely result in a damaged partition. All other fails will probably leave the original partition intact and thus the device can be rebooted. So far my understanding.
General hints for RUUmode / Download Mode zips
- Opening a zip is best done with 7zip as WinRAR and other zipping tools have lead to flash fails in the past.
- Choose low compression, higher compressions often fail. Pick "save" or "normal" to be safe, anything higher could cause the unzip in Bootloader to fail.
- Adding and Removing images is not a problem. The naming of the partition images seems flexible, yet if you encounter an Error 23: parsing image fail you need to rename the relevant image to something stock as not all names seem to be recognizable. The Hboot/Aboot determines the right partition from the header inside the image.
- Additional Dots in zip file names are known to have caused issues for a few people.
- Spaces in names are a no-go!
- Custom Recoveries can be added to those zips as well as custom kernels. In fact, if your phone is S-OFF, you can pretty much add anything and name it e.g. recovery.img and it will be flashed. You gotta be very very careful, as this is an easy way to break your phone. Make sure not to mess around with modified images!
- With S-ON, those zips only flash if everything is totally stock, from the android-info.txt being right up to all images being the correct versions for that update package and all having the right signatures. Reads: no custom messing with firmware zips for S-ON phones. In fact: apart from HTC OTA firmware.zip's and RUU's, nothing will flash with S-ON at all...
General hints for android-info.txt
- Use an Editor that doesn't mess up linebreaks like Windows Notepad does. Use
Notepad++
- MID's can be added one per line. Also supports wildcards i think e.g.: 71******, but i'm not sure.
- CID's can easily be added or removed- one per line, definitely supports wildcards (used by HTC in DevEd phone)
- Mainver line: should hold the version of the used set of firmware images. Example how to format the version: 2.24.401.1 (2= Base version always increases by 1 with each Android base version rise, 24= Build version from HTC, 401= Regional/Customer identifier, 1= Revision of the HTC Build). This line is being written to the /misc Partition and is meant to reflect the whole phones software version - it is not meant to only describe the firmware part or the ROM part alone. HTC has intended the Version to always represent the whole thing, firmware version matching the ROM version. therefore, it would be wise to always run matching firmware and ROM versions, except where explicitly recommended otherwise. Mismatches can cause anything from no issues over radio problems up to semi-bricks.
- hboot pre-update line: usually says "3" but i have seen different numbers. I think they determine if hboot-preflash is required (when you get Error 90 - please flush image again immediately this is when the hboot/aboot needs to be flashed separately first and then the rest. If you encounter this, you need to run the flash command you just did, again.
- btype:1 not clear. [Item subject to change]
- aareport:1 Since HTC hboots/aboots, boot and recovery images come as "hboot_signedbyaa / aboot_signedbyaa / boot_signedbyaa and recovery_signedbyaa i would read this as "aa" representing htc ("hboot signed by aa"). It could possibly mean check on the signature in hboot/aboot/boot/recovery - all of those also come in unsigned flavors - in HTC OTA's, those are usually without the _signedbyaa but in the RUU, they are carrying a signature). So, aareport: 1 can just mean check on signature yes or no.
- Delcache means erase cache when rebooting. Simple. Some firmwares seem to need it, some don't. Line is not present in every android-info.txt. If you mess with a zip that contains the line, leave it active. This is also not referring to the Android OS cache partition. It refers to the separate Kernel and Recovery Cache. Sometimes, not deleting Kernel or Recovery Cache after flashing those leads to malfunctions. If the Kernel is launching and there is an older conflicting copy cached, the phone won't boot past Kernel stage (before the bootanimation starts), if Recovery is conflicting with a cached copy (usually after flashing a new/different recovery), it will lead to the recovery not booting or malfunction (like aborting an ongoing ROM flash or not being able to execute other functions).
RUUmode:
is the mode used for RUU flashes by HTC. It allows a few more things than the normal fastboot. You recognize it by looking at the phone's screen. It will be black, showing only a silver HTC logo and if a command is being active, a green progress bar. New M9 RUUMode now shows a percentage counter below the bar.
Download Mode:
New flashing mode, introduced 2015 with the HTC One M9, due to changing to a different, aboot-based bootloader structure, HTC also introduced the use of an HOSD partition containing sort of a micro-linux with extended fastboot-capabilities. It provides much more logging output and can be considered the better flashing environment now. When flashing my firmware.zips, I recommend using Download Mode over RUUMode, as it gives you much more feedback. The ARUWizard (a.k.a RUU or FUU) might still expect RUUMode for flashing, but when manually flashing or also when using SDCard method, Download Mode will work best.
Bootloader Mode:
You can also directly flash most firmware partition images with the by-name method, and that works only when booted to Bootloader (the white screen with colored text lines). In this mode, you can for example flash stuff like this: htc_fastboot flash recovery recovery.img or htc_fastboot flash adsp adsp.img - almost all image files can be flashed separately without HOSD or RUUMode in bootloader.
On the HTC10 this has become a critical function when recovering from flashing frankenbuild-firmware. After the Android N transition for example, a combination of old keymaster.img with new Android N firmware would lead to broken Download Mode and broken RUUMode, hence disabling all flashing methods. Using this direct image flashing method, you can recover your phone in such a situation. Inside the firmware.zips will be a file called partition_info - you can open that with a text editor like NotePad++ and see all the flashing names for the partition and this way figure out how to reflash every single partition manually.
NOTICE: do not flash aboot_signed.img this way! The only image that should not be flashed directly over itself when booted into bootloader!
Recovery flash risk:
Some of you might have heard of, or are thinking about flashing firmware using recovery.
Although it is perfectly possible to write firmware images to the NAND chip using the DD method in recovery (either with a script or by using ADB shell dd) it is highly recommended that no developer employs this method (except if it's the only way to rescue a damaged device, e.g it only boots to recovery or something like that). This suggestion can be limited to boot critical files (SBL images, Aboot, HOSD, Keymaster for instance), but I prefer to see this as a general good practice thing. The reason behind this is, that DD has no inbuilt write verification. If there is just one single bit that does not get written correctly, DD won't notice and won't correct it. With some bad luck, you end up with a brick this way.
JTAG with a RIFF Box
Every device of these days has so-called jtag test-points. Basically, these are points on the mainboards, where a direct connection to the main chip can be established and then that chip can be read and written to with an external device. Sometimes, these testpoints are hidden (like they are normal contacts of the chip) and no direct visible gold points on the board. It always takes a while after a device is released until the jtag layout is fully discovered but once that is done, companies like multi-com.pl start manufacturing small boards with pins that can be pressed onto the mainboard, so no soldering to the device is required. Once such a board exists, the mainboard can be hooked to the RIFF box which can rewrite a dead chip from the outside.
As long as there is no such small board (called a "JIG") the phone can still be revived but it is necessary to solder hair-thin wires to the test-points. That is perfectly possible, Tecardo can do such a thing, but its not very good for the board and cannot be done very often. At some point the solder points will degrade so much that the board is garbage then.
In case you really brick your device, you can contact Tecardo here:
http://xdaforums.com/showthread.php?t=2116062
MID and CID
MID = Model Identification. It serves the purpose of identifying the Model of the phone. There usually are several different ones. The ModelID in android-info.txt is CaSeSenSiTivE!
Some limited Data is here:
https://docs.google.com/spreadsheet...ShfYNFAfSe-imhhqtVfeMPVDA/edit#gid=1606643937
CID = Customer ID and describes, for which customer HTC made this phone. HTC has a few own CID's for its regional stores. Then certain carriers decide to have their own CID. Some carriers even have their own Model ID's.
So, while the MID more like describes the hardware, the CID basically just describes the software set that comes delivered with it. Both get checked on when flashing in RUUmode. How to trick this system? Fairly easy. Just add your respective MID or CID to the android-info.txt file inside the ZIP or make your phone SuperCID (My Batch Tool can do that automatically - but remember: all this only works on S-OFF phones).
S-OFF:
S-OFF refers to the NAND's security lock. S is for security and OFF means the security is switched off. The factory state HTC's phones ship with is ON, except for the userdata partition, which of course is always unlocked.
The key for that lock is the most heavily guarded secret in HTC's software vaults. It cannot be extracted, bought or otherwise obtained from them. There is no official way to unlock the NAND partitions (approximately similar to what Apple fans do when they jailbreak their products, although technically not quite as similar). While the HTC Dev Unlock (available through htcdev.com) just unlocks 3 partitions (Boot, Recovery, System), the S-OFF hack we use unlocks all partitions, thus enabling the flashing of custom, modified or other devices firmware. This is what you want for this thread and you can get it from the famous reverse engineers Jcase and Beaups over at:
http://theroot.ninja/ or alternatively purchase a Java Card and learn how to work it, from chinese sellers on Alibaba, sometimes Ebay. Then there is a way to do it with an XTC Clip. But SunShine S-OFF is by far the safest and fairest method. You will only be charged if it works and the guys over at sunshine are really helpful.
A more detailed look at how S-OFF works
[Subject to change - not a definite explanation, just how I think it works]
In the phone's Firmware is a component that checks if certain partitions have a digital signature from HTC and deny write access if the signature is wrong or missing. The checking component is known to be the Security, which can be set to OFF. Then we say the phone is
S-OFF.
System, recovery and boot do not get signature checked at all once you unlocked your phone on htcdev.com. The other partitions however do get checked as long as Security flag is set to ON. Partition 3 is where the Security flag is located and maybe also the checking routine that checks the other partions digital signatures,
The S-ON state is resembled by a 3 in the fastboot command to switch security on. It is: fastboot oem writesecurflag 3. You do NOT want to do that while any custom firmware is running. Only after a full RUU that removes any modifications.
Why? For some partitions like the splash screen, it might not lead to a brick if you set security to ON while a custom splash is installed (then failing the signature check), as this partition is not vital for the boot process, it might just be skipped and give you an error message (I have never tried obviously). Other partitions however, boot critical partitions like Hboot/Aboot.... You guys have to understand that altering any of these partitions can be deadly to your phone if you happen to leave them altered when switching security back on.
Determining your Firmware Version
I believe there is some wrong info circulating the HTC Fora. People keep saying when running fastboot getvar all it will report the Firmware Version in the line Version-Main. This is not always true though. Fastboot getvar all or alternatively getvar mainver pulls a version it finds in the MISC partition and relies on that to be correctly updated.
Source
So how does that version string get updated? It is being taken from the android-info.txt file in any firmware zip that you flashed. The last zip you flashed determines what will be reported by the getvar function. So if you mess around with Firmware.zips and RUU's a lot, chances are, that the version reported there is not equivalent to what you are already running. Often the android-info.txt has version entries not appropriate for the actual zip contents, for compatibility reasons, because it wasn't done properly or whatever. My zips usually have the correct MainVer though.
The "Firmware" as a concept like we use it on XDA does not exist in HTC's terms. HTC does NOT differentiate between the /System Partition (what we know as "the ROM") and the other 36 partitions. Hence, if you run getvar all or getvar mainver on a stock phone, it will report correctly. It does not go looking for a fictitious place where it would find a separate "Firmware" version. That place it is looking at is the Misc Partition and that's correct as long as you haven't messed with lots of different Firmware zips... So, if you happen to run a hybrid system with a ROM from one base and the other partition images from another base or multiple bases (like hboot from 1.27, radio from 4.06 and ROM from 3.62) the getvar function will report as "Version-Main" what it finds in /misc/, precisely the last zip you flashed determines the string put there.
Example: you flashed a radio with a RUUmode zip from Base X.YY but the android-info.txt is maybe still an old one because the dude who made the zip, just dropped the new radio into an old existing zip, the getvar function will later report that old version as your mainver.
To check your firmware: boot to bootloader and look at the combination of hboot version and radio version - if you didn't flash those separate, the combination will let you know what base you are on (each OTA and RUU has the radioversion in its name).
Finding out your firmware is a game of guesses and knowing what you did to your device and where you are coming from.
If totally lost, best thing is to reflash some clean stock package to be sure you are on the same level with all partitions.
Long story short: you better know what you do because finding out your firmware is going to be difficult if you don't.
Red Text removal
Back in 2012/13 with the HTC One M7, I wrote a guide on how to modify Hboots/Aboots so that they won't show the red warning text on the bootsplash (
here). The following years I mentioned in many many posts, why this is a dangerous idea to do. During the course of the past years, I then came to the conclusion that it was exceptionally stupid of me to provide these instructions in the first place. Many people bricked their phones while using a modified bootloader, mostly when going back to S-ON and then forgetting to replace the modified bootloader with a stock signed bootloader first. The device won't boot anymore if that happens. It will be dead. The NoRed aboots are quite a considerable risk and justify in no way the split second of boottime without red text. I strongly advise against using them.
Also, I do not want any discussion, promotion etc. of NoRed Aboots in my thread.
Further reading
If you happen to get access to an international RUU, share it with Alex, or me please. RUU
's are hosted on
androidruu.com by Alex from Androidfilehost.com within a short time after being made available to him. Hit him up on Twitter with a link and ask him to add it or send it to me and i will!
