HBA software features in Redfish

This is a familiar problem with RDE. The schema exposes a bunch of properties, some of them can be applied by the firmware, others are host-software-level constructs that the firmware is completely oblivious to. While the request is going to the FW via RDE, the change would have to be applied by the SW.

First, I'd recommend to start by supporting properties that your FW can actually support. Note that virtually all properties in the schema, not counting housekeeping ones, are optional. If there's no clear way for your FW to support nuanced RAID, the most immediate solution would be to skip those properties.

A more permanent solution I'm afraid would involve building some sort of proprietary FW<->SW bride that interrogates the SW about things beyond FW scope. All of this would be 100% proprietary, and from DMTF's perspective, your own internal implementation detail.

This seems to be a common pain point for devices that want to "bolt-on" RDE on top of an existing stack. The classic SW/FW hierarchy assumes SW is the master and often, the sole source of configuration changes. SW talks, FW listens. FW issuing OS-level changes through the SW inverts this model.

An idea I've been floating around is a separate companion app that provides SW functionality to RDE FW, acting as a standalone proxy.

Redfish went into broad deployment onto BMCs, and BMCs used whatever means at their disposal to build their Redfish Resources. AFAIK that involved a multitude of various tools, protocols, glue code, proxies.
For example, on some servers with Redfish, for non-RDE devices, the BMC still exposes a lot of data about the network interfaces. It does so in part through a software companion app running on the host OS. While I haven't ever investigated it's function thoroughly, my understanding is that under the hood it does the equivalent of lspci/netstat/other cmd-line tools, and pushes that to the BMC through some channel. Data for Resources any BMC exposes is being AFAIK harvested and applied through a mix of whatever protocols the managed device in mention had available - NVME-Mi for SSDs, NC-SI for NICs, PMBUS for PSUs... The burden of integrating all that was on the BMC, or more generally, the device that hosts the Redfish service. With functionality like PropertyValueConflict annotations for when the user gives conflicting orders, or mutually exclusive states, a Redfish service that relies on external protocols would have to have extremely deep knowledge of the device it's managing. It wasn't practical or sustainable, and thus RDE was born to allow the device to take ownership and interact with Redfish directly.

I do get your point about the FW being unaware of stuff like topology. Same thing on networking side, where the HW/FW talks in terms of frame size and neither knows nor cares how those bytes are divided between user-usable data payload (MTU) and headers.

I believe there's quite a lot of complexity involved in a generic solution - different OSes, multi-host environments, deployments without a host OS, various levels of virtualization.. a generic fits-all solution might be very complex, incomplete or only temporarily complete. Add to that, Redfish is a model that's supposed to be draped over multiple types of devices. A fancy, smart Redfish-enabled network switch, for example, with it's single-purpose, locked-down OS, has access to both OS-level stuff and registers.




I don't get this part. If the firmware is locked down, then presumably you can't fit RDE onto it either?




Nope, not unless the BMC vendor builds some custom solution.
Although in your case, I believe the chain should be BMC->FW->SW->FW->BMC, not a parallel BMC->SW chain. The situation where both the SW and FW manage parts of Redfish would get messy quickly, especially on a per-property level. Imagine a PATCH payload contained two properties, one that is applied by SW, other by FW, and they are in conflict. Who resolves? 

Your FW would decide what PDRs it serves, what Resources it exposes, what operation types it supports etc etc - as a single source. And when needed, the FW would reach out to SW for help. This way you don't need the BMC to do anything special. From the BMC's perspective, it's just talking to FW. And RDE does support long-running tasks and delayed ETags in terms of timing, so who the FW talks to becomes an "implementation detail".

The issue of wanting to expose extra HW functionality not covered by the standard driver model isn't new. Any graphics card you buy nowadays has a companion app for playing around with LEDs, fan curves and whatnot. I'm thinking that the next step is a similar RDE bridge utility, that proxies requests from an RDE FW device into the SW. Make it fully optional, make some Resources/Properties require this app with a clean fallback into null/skip + info that you need to install the app to have full RDE/Redfish functionality. 


Note that there's two FW to host OS connections - the blue regular, existing PCIe mechanism for in-band configuration, shuttling data etc, and the new PCIe to app connection. 

My guess is the earliest DMTF can try to come up with a standard for this is once there's one or more such custom solutions to draw lessons from. 

Relying on the OS isn't ideal. If I mess up some settings on my storage controller, I presumably can lock myself out from the OS booting. No matter how broken/misconfigured the OS might be, OOB manageability should be available for at least diagnostics, if not recovery of operational settings. If the host OS is up then, practically speaking, SSHing onto it in-band and managing it that is faster and more direct than Redfish. In other words, all else being equal, a device that for it's participation in Redfish, requires the OS to be up is less useful than a device that one that does not.

You will probably run into issues with 3 - Redfish requires it's settings to be persistent. I understand that the device non-volatile memory (FW *and* settings) are static? That would preclude any changes, and make the Redfish model RO.

We'll have to compare notes on the proxy solution one day. I do believe the RDE-into-OS is a design gap that sooner or later will need to be addressed somehow, but someone needs to make the first mistakes.

One thing that can be worked in the meanwhile on is grouping properties into relevant schemas. On the networking side for example, the schemas are pretty well organized into those that can be fulfilled by HW/FW alone, and those that need to be populated by SW. The natural direction this leads towards on NIC side is that the proxy owns entire (high-level, software-centric) Resources - is it like that in storage schemas, or are you finding that firmware and software related properties are mixed into one schema?