I am that guy on Reddit. Here is the definition of J_DBG_EMMC. I can't remember clearly whether the VCC and GND at the top-right corner are labled in the right order. So please make sure the GND pin is short to other GND before soldering. I will upload a "Good" emmc image of idrac soon .

I have 4 MOBO of R720 with the same problem. This one in the picture is fcuked up, you guys could see some solder pads were damaged. That is suspicious all 4 mobo died in the same way, and all of them are acting like EMMC damaged and cant be Written into. It shouldn't be the flaw of the emmc chip. Instead, I guess, it's the issue of Dell's design.

We can read this chip on borad with a SDCard reader, however, I am still searching a way to cut off the connection of the SoC to EMMC chip. Very annoying, when trying to write data into the emmc, SoC interferes so heavily. Someone suggested get the quartz oscillator which to the right edge of the EMMC offline. I am some kind busy these days, if it works please let me know.

Thanks to the help here, I was able to recover iDRAC on my Dell r620 and update it to the latest version.


Initial State
I purchased a r620 that had iDRAC version 1.66.65 installed and all update attempts failed with the following error message: RED007: Unable to verify Update Package signature.

After trying to update using multiple methods (tftp, racadm, lifecycle controller) I got to a state where it looked like the update was going to take, however when iDRAC reset, it failed to initialize with the fans running at full throttle and the iDRAC status light off.


Connecting to iDRAC UART
I soldered a 4 pin header to J_IDRAC_UART, then connected it to the Raspberry Pi GPIO on GND, TX, and RX. (Any UART controller should work, I just had a RPi available)
Note: If using a Raspberry Pi's serial port, you may need to disable the Raspberry Pi uart console

After connecting to the serial port with "screen /dev/serial0 115200" and turning on the system, I noticed multiple errors relating to SQUASHFS

Dec 31 18:00:53 (none) kernel: SQUASHFS error: zlib_inflate error, data probably corrupt
Dec 31 18:00:53 (none) kernel: SQUASHFS error: squashfs_read_data failed to read block 0x18337f9
Dec 31 18:00:53 (none) kernel: SQUASHFS error: Unable to read fragment cache entry [18337f9]
 

U-Boot was loading the iDRAC image, however, iDRAC would not start due to corruption.


Interrupting U-Boot
To recover the firmware, I had to force U-Boot to interrupt the boot process. I did this by bridging the second pair of pads in SW_IDRAC_DBG.
Note: You have to be fairly fast with this and press any key when prompted to interrupt the boot process. To assist with this, you can bridge the first pair of pads, which forces iDRAC to reboot, then move the wire to the second pair.

Hit any key to stop autoboot:  0
WDT2: Disable in abortboot()
OSWDT: Disable in abortboot()
CPLD_BMCRDY: Enable BMC_MIN_RDY in abortboot().  Prevent BIOS reset.
 
NOTE: After stopping u-boot in this development mode.  You may need to
      warm/cold reset the server when booting iDRAC manually as BIOS
      may have already viewed iDRAC as unresponsive.
 
iDRAC7=>
 

After getting to the iDRAC7 console, I attempted to recover the firmware using tftp as @ldv did, but I could not get U-Boot to communicate over my network. Instead, I recovered the firmware using a SD card.


Preparing the SD Card
1. Format the SD card as FAT, then copy firmimg.d7 from the iDRAC update exe to the card. I formatted the card with a 256MB FAT partition, but this may not be necessary.
2. Insert the SD card into the server.
3. Check if uboot can read the SD card

iDRAC7=> mmc list
sh_mmcif: 0
 sh-sdhi: 1
iDRAC7=> mmc rescan 1
iDRAC7=>
 

4. If you receive the error 'Card did not respond to voltage select!', try rebooting iDRAC with the SD Card already inserted.


Recovering the Firmware
I was now able to recover the firmware from the SD card using the following command:

iDRAC7=> util recover -emmc -from sd -f firmimg.d7
 

After the partitions are updated, iDRAC should reboot. iDRAC hung for me at 'Monolithic/DRB' and I had to remove the SD Card and hold the iDRAC reset button to get past this point. iDRAC was then working correctly and I could boot the system without issues.


Other notes:
-I recovered the firmware to 2.10.10.10, then to the latest iDRAC version using this method. You may be able to jump straight to the latest version instead.
-After updating iDRAC, I was still unable to update it using the traditional methods. Traditional updates continued to fail with RED007: Unable to verify Update Package signature.

Zitat von ldv im Beitrag #83

---

״Alternately I'm wondering if there isn't a way to manipulate the settings in the idrac eprom to boot from the sdcard so that it's persistent between boots. ״ @willard
It definitely possible. Actually, that’s what I’m going to try first, before the attempt to replace the EMMC chip.
I think it can be done by setting the environment variables. What I figured out, is that these variables can be not only a “key = value “ but also a “key={script }” type.
So you can try to run following(excuse me if the syntax is wrong, since I bricked my iDRAC again I can’t check it at this moment , so I’m typing it right out of my memory):
1. To set SD as boot device permanently
set bootcmd ‘sd_boot’
saveenv

2. To boot from tftp permanently
set bootcmd ‘tftpboot -f firming.d7; go addr 0x81000000; bootm;’
saveenv

---

set bootcmd ‘tftpboot -f firming.d7; go addr 0x81000000; bootm;’
 

. Anyways, I now cannot seem to interrupt U-Boot anymore via the SW2 pins. It's now stuck downloading the image via tftp, then doing something called "## Starting application at 0x000..." (not 0x81000000) and immediately restarting U-Boot, seeming like it panics or something. It's now doing this forever. How can I restore the environment or switch to the backup env? I don't have an SD card reader.
I tried tricking it by providing an invalid file or something, but it keeps retrying until forever...

EDIT: I managed to escape the boot loop by finding the relevant clue inside the U-Boot documentation: if bootdelay=0, then you're going to have a hard time interrupting u-boot. Fortunately, you can glitch yourself into u-boot anyway using CTRL-C. Just be sure to hold it down when rebooting and with a bit of luck you either magically enter u-boot or you corrupt the memory of uboot1, so it will override it with the content of uboot2.

Unfortunately, tftp firmimg.d7; go 0x81000000 doesn't work either. It will just infinitely boot loop without actually booting.

I'm facing a similar issue: iDRAC won't boot completely, stuck in this:

syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:21:28 idrac8 syslogd 1.4.2: restart.
syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:27:09 idrac8 syslogd 1.4.2: restart.
syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:29:04 idrac8 syslogd 1.4.2: restart.

I connected the serial console successfully, but cannot break out of U-Boot boot process. Tried shorting second pair of pins, tried CTRL+C, without any luck. Actually spamming CTRL+C results in the boot loop ocassionally, but no U-boot prompt... Anyone has any other ideas how to solve this?

Zitat

---

-After updating iDRAC, I was still unable to update it using the traditional methods. Traditional updates continued to fail with RED007: Unable to verify Update Package signature.

---

Isn't that error due to dell replacing the signing certificate in 2.21.21.21 (i think it was) I had that error updating until I checked through the releases and updated to the previous important update then I could update,

Thanks to everyone who persisted with this and managed to get to a point where the iDrac can sometimes be recovered. I have read all of the replies on this forum and was really encouraged when @doomguy successfully recovered his iDrac. I followed his instructions to the letter on an R320 but I couldn't get the firmware to write to the emmc. I had exactly the same errors as others:

iDRAC7=> util recover -emmc -from sd -f firmimg.d7

UTIL RECOVER:Transport:sd TargetMMC:EMMC File:firmimg.d7
Loading file "firmimg.d7" from mmc device 1:1 (xxb1)
62953181 bytes read
UTIL RECOVER:SD load passed from EXT2 fs.
UTIL RECOVER:Transport time [sec:mil]: 01:08:542

Clear OS images in partition/s.
Clear kernelN, rootfsN, ubootN
'EMMC' blk size=[0x200][512] Erase to 0xffffffff
Mem buf size=[0x06000000][100663296] Total bytes=[0x08000000][134217728]
blocks[0x1:0x40001][1:262145]
mmc write failed000] Buf[0x88000000]..fill buffer..erase/write
util flash:
Failure writing blocks [1:196609] RC=0xffffffac
UTIL PASS

I tried versions 1.4, 1.5, 1.6, 2.1 and 2.6 and they all failed with the same error. Did anyone manage to solve this last problem?

I'll keep trying as I have nothing to lose.

I have now come to the conclusion as many others have that the eMMC gets corrupt when the firmware update fails. How or why I don't know. The only way to prove this would be to replace it but I don't have the skills or equipment to do this.

What I was able to find, which may help others is the uboot recover method used by @doomguy does work in some cases as I tried this on a fully working iDRAC without issue.

duplicate post, ignore

Zitat von haimgelf im Beitrag #94

---

I'm facing a similar issue: iDRAC won't boot completely, stuck in this:

syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:21:28 idrac8 syslogd 1.4.2: restart.
syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:27:09 idrac8 syslogd 1.4.2: restart.
syslogd: /mmc1/idraclogs: Read-only file system
Dec 31 18:29:04 idrac8 syslogd 1.4.2: restart.

I connected the serial console successfully, but cannot break out of U-Boot boot process. Tried shorting second pair of pins, tried CTRL+C, without any luck. Actually spamming CTRL+C results in the boot loop ocassionally, but no U-boot prompt... Anyone has any other ideas how to solve this?

---

Zitat von doomguy im Beitrag #92

---

Thanks to the help here, I was able to recover iDRAC on my Dell r620 and update it to the latest version.

-After updating iDRAC, I was still unable to update it using the traditional methods. Traditional updates continued to fail with RED007: Unable to verify Update Package signature.

---

This will probably be my final reply to this forum as I somehow managed to completely kill the motherboard so any hope of recovering the iDRAC are now pretty much zero. Here are some of the findings that I made during my testing:

On my R320 I couldn't get the network working, I discovered that it was using the wrong Ethernet device. So to connect to the TFTP server I did the following:

setenv ethact sh_eth.0 <- Sets the Ethernet device to eth.0, this the Ethernet port 1 on the motherboard
setenv serverip 192.168.0.x <- This is the IP address of the TFTP Server you want to get the file from
tftpboot 0x81000000 192.168.0.3:firmimg.d7 <- This gets the image file from the TFTP server and loads it into memory

The following never worked for me:

go 0x81000000 <- This would boot the iDRAC using the existing firmware
bootm <- This would boot the iDRAC using the existing firmware

The last thing I did before the motherboard failed was:

iDRAC7=> fwu dev 4

No. Device
---------------------
1 EVB SD/MMC
4 FLASH

Current Device ==> 4

iDRAC7=> fwu update 0x81000000

*** Updating Partition 5 ***
Checking image header CRC ... OK
Checking platform env ID..... OK
Checking kernel image CRC ... OK
Checking rootfs image CRC ... OK
Checking u-boot image CRC ... OK
Skipping u-boot update ...... NO
Checking Platform image CRC . OK
Copying kernel image ........ OK
Copying rootfs .............. OK
Copying u-boot1 to flash..... OK
Copying u-boot2 to flash..... OK
Copying u-boot to MMC........ fwu_block_write()DEV_FLASH:util_spi_erase(): SPI f

Fail
*** fail to update u-boot image on SD card ***
*** fail to update image ***

My final conclusion is that the eMMC had failed which caused the original iDRAC firmware update to fail so no matter what you do you'll never be able to reflash the eMMC.

If you have a power cut during a firmware update then I'm pretty sure the UART recovery method will work just fine.

Hello,

I have a similar problem with iDRAC7 on one R620. The server is starting normally, 'configuring memory' and then stucks at iDrac initialization. After long wait time it writes 'iDRAC initialization failed' and reboots. I soldered the UART pins and i got serial output (https://pastebin.com/mGfLaYmB). It stops at systemd[1]: Failed to insert 'autofs4' and after long wait shows the reset of output and stucks there.
I also tried to jump the A2,B2 pins in the sw_idrac_dbg region but it not disrupt the u-boot process.
Can you provide any insight about this?

I have a Dell PowerEdge R320.
EMMC was corrupted during firmware update.
I soldered to the debug port and connected to it through the adapter on max232.
During boot, Idrac experienced multiple errors in the terminal.

How did I get the download to go from the SD-Card instead of EMMC:
1. I removed the SPI-flash and made a backup of the firmware, sealed it back.
2. He closed the contacts A2 and B2.
3. Set up TFTP, put the file firmimg.d7 (I took 1.31.30 from Idrac, originally 1.30.30), install the SD card.
4. In the terminal, you need to interrupt the download - press any key. You fall out in iDRAC7 u-boot.
5. iDRAC7 => util recover -sd -from tftp
6. Reboot, you may need to restore the SPI firmware from the backup.
7.iDRAC7 => set fwummc sd
8.iDRAC7 => saveenv
9. Made a byte image of the SD card into the sdimage.img file (using WinHex).
10. Saved the second section to a file (SquashFS file system) part2.img.
11. In Linux, unsquashfs ./part2.img
12. Replaced the line mmcblk0 with mmcblk1 in the files
./squashfs-root/etc/sysapps_script/S_3160_create_corepart.sh
./squashfs-root/etc/sysapps_script/S_3160_create_master.sh
./squashfs-root/etc/sysapps_script/S_3160_create_scratch.sh
./squashfs-root/etc/sysapps_script/config_lib_init.sh
13. Gathered a new image of SquashFS
mksquashfs ./squashfs-root/* part2_new.img -b 1048576
14. Copy sdimage.img to a new sdimage_new.img file
15. Replaced in the sdimage_new.img file the second section from the file part2.img.
16. I write sdimage_new.img on the SD-Card.
17. Insert the SD-Card into Dell. When booting, u-boot reported invalid partitions on the SD-Card, but the download continued.
18. Idrac works - it's linux.
19. Ура!

20. I also wanted to read emmc for analysis.
21. On Idrac-linux: ftpput -u ftp_user -p ftp_pass your_ftp_server /ftpdir/ftpfile /dev/mmcblk0
22. With the resulting image, we do the same with 10 points.
23. If necessary, repair the file systems, I had to completely replace /dev/mmcblk1p6 from/dev/mmcblk1p2
24. We get a working idrac.

Sorry for my bad english. It's google-translate.

images
ibb.co/XV3dgdC
ibb.co/SPtkZQy

ldv, willard, did either of you ever replace your emmc? I'm having similar issues though I seem to have an SK Hynix H26M31001FPA part. And I'm struggling just to interrupt the u-boot to attempt a tftpboot.

Zitat von oso2k im Beitrag #104

---

And I'm struggling just to interrupt the u-boot to attempt a tftpboot.

---