slotain
Minnow
It's bigger than a regular hat. It's funny.
Posts: 19 
|
Post by slotain on Oct 27, 2021 21:54:28 GMT
TLDR: What profile is used to validate the Redfish-Profile-Simulator to 100%? Where can I find the Redfish work-in-progress profiles (mentioned in the profile bundle doc)? I want to build an amazing Redfish service right out of the gate: Am I doing it right by attempting to validate against a profile as a first step?
This post ended up being longer than I expected. If I have time to edit it, I'll make it shorter.
I'm using the git@github.com:DMTF/Redfish-Profile-Simulator.git and in the documentation for it, it states that it's aligned with the OCP Remote Machine Management Spec.
After looking for a while, and watching youtube for the past couple days (great presentations by the way, jautor!), I found this website
Anyway, I expected to see 100% passing (no problems found) for one or the other OCP profiles that I downloaded from that site (opencompute's site, above). The two profiles that I attempted to run against the git@github.com:DMTF/Redfish-Interop-Validator.git seem to only report less than ideal results
$ python RedfishInteropValidator.py -c config/example.ini profiles/OCPServerHardwareManagement.v0_2_3.json
Redfish Interop Validator, version 2.0.1
... some passes ... some failures ...
Validation has failed: 6 problems found
$ python RedfishInteropValidator.py -c config/example.ini profiles/OCPBaselineHardwareManagement.v1_0_0.json
... etc. etc.
Validation has failed: 9 problems foundI'm also a bit lost with respect to the Redfish Profile Bundle, because it says there are supposed to be folders in the zip file, but I don't see folders. And the profiles in this bundle don't seems to validate using the Redfish-Interop-Validator against the Redfish-Profile-Simulator (not even a little bit, they are bugged or they're not actually profiles but schemas).
Where can I find the work in progress profiles?
There's a bunch of profiles I can test here as well, from opencompute. Maybe the holy grail is in here. I'll report back.
[... Later]
So, I edited the config/example.ini to point to the latest running instance of the Redfish-Profile-Simulator and I cloned the HWMgmt-OCP-Profiles
Some good results, but none are 100%. Maybe it's time to dig into why each of them are failing and consider that it's my fault... (slight panic)
Redfish-Interop-Validator $ for x in ../HWMgmt-OCP-Profiles/* ; do echo Doing $x; python RedfishInteropValidator.py -c config/example.ini $x ; done | grep -e Doing -e Validation
Doing ../HWMgmt-OCP-Profiles/OCPBaselineHardwareManagement.json
Validation has failed: 5 problems found
Doing ../HWMgmt-OCP-Profiles/OCPBaselineHardwareManagement.v1_0_0.json
Validation has failed: 9 problems found
Doing ../HWMgmt-OCP-Profiles/OCPBaselineHardwareManagement.v1_0_1.json
Validation has failed: 4 problems found
Doing ../HWMgmt-OCP-Profiles/OCPDoorHeatExchangerHardwareManagement.v0_0_5.json
Validation has failed: 10 problems found
Doing ../HWMgmt-OCP-Profiles/OCPRackManagerController.v1_0_0.json
Validation has failed: 26 problems found
Doing ../HWMgmt-OCP-Profiles/OCPServerHardwareManagement.json
Validation has failed: 6 problems found
Doing ../HWMgmt-OCP-Profiles/OCPServerHardwareManagement.v0_2_3.json
Traceback (most recent call last):
... snipped out ...
IndexError: tuple index out of range
Doing ../HWMgmt-OCP-Profiles/OCPServerHardwareManagement.v0_2_4.json
Validation has failed: 6 problems found
Doing ../HWMgmt-OCP-Profiles/OCPStorageManagement.json
Traceback (most recent call last):
... snipped out ...
TypeError: 'NoneType' object is not iterable
My motivation for doing this work (attempting to use the validator) is because I would like to know that my next Redfish service (my first) will meet some standard. I found the profiles to be a good goal to shoot for, but tell me if I'm being too bold. I don't know if I should be so interested in 100% completeness. I have a Dell IDRAC system in my lab, and I'm executing a recent profile against it. There are quite a few failures. I'll report back with results.
|
|
|
|
Post by mraineri on Oct 28, 2021 12:22:57 GMT
To be honest, the Redfish-Profile-Simulator is quite out of date at this point. Originally the DMTF held the work for guiding OCP for Redfish requirements, which included drafting the original requirements specification and creating that mockup tool to help OCP understand Redfish. Since then, OCP took over that work and is more directly involved with crafting Redfish requirements for their use cases, which includes the profile creation. You're right in that the OCPServerHardwareManagement is the starting point, but new requirements have gone into the OCP specifications and the profiles have been updated as such, so the existing Redfish-Profile-Simulator will likely not pass validation.
jleung would be interested in the results you collect. There are some examples of requirements in the OCP profile that I think are too strict at the moment and need to be addressed by OCP. For example, the OCP server profile currently requires fully fleshed out EthernetInterface resources (IP address management and everything) for both the BMC and the host system. I think such a requirement is very reasonable for BMCs, but I don't think it's realistic for host systems in all cases, unless OCP is enforcing tight coupling between the BMC and host system to trade information. |
|
|
|
Post by jleung on Oct 28, 2021 15:33:39 GMT
Thank you for attempting this testing. Note that...
The profiles approved by the OCP IC are listed below. There is a corresponding User Guide (in the contribution database) which describes the user interaction with the Redfish interface.
- OCPBaselineHardwareManagement.v1_0_1.json
- OCPRackManagerController.v1_0_0.json
The profile which will be seeking OCP IC approval
- OCPServerHardwareManagement.v0_2_4.json which will be rename v1.0.0
(I will file Mike issue mentioned above, so the Server project can weigh in)
The profiles which have been proposed
- OCPRackManagerController.v1_1_0.json
- OCPDoorHeatExchangerHardwareManagement.v0_0_5.json
The profiles which are obsolete or superseded are
- OCPServerHardwareManagement.json
- OCPServerHardwareManagement.v0_2_3.json
- OCPBaselineHardwareManagement.json
- OCPBaselineHardwareManagement.v1_0_0.json
I will propose that the structure of the repository be changed to make the state of the profiles more obvious to the casual user.
Thanks,
John
|
|
slotain
Minnow
It's bigger than a regular hat. It's funny.
Posts: 19
|
Post by slotain on Oct 28, 2021 15:38:07 GMT
The results are already edited in the original post, near the end of my post where I grep for "Validation" -- jleung have a look there. I certainly didn't want to fill my post with garbage text, but I can just as easily link to a gist with the full text dump. Here's that for you: gist.github.com/activedecay/87afac1e0e57e3e309a662b9352b183b which I added some comments about how I created the output (it was trivial). That's interesting, mraineri , that you have some opinions about how the profiles could be used with respect to BMCs, because that's going to be my first and hopefully awesome Redfish service. I work for a systems manufacturer and we're building a new BMC to manage the new chassis with a computer system, backplane, and expansion-graphics.
I hope that found the right forum to continue more discussions about this implementing a Redfish service from scratch. From what I gather, after watching many hours of YouTube and reading as much as I can find, it seems as though when my boss says, "implement a Redfish service", that in truth I can sort of pick a depth and breadth to that as I see fit. I wonder if the old "implement what you have" is still a pretty decent way to proceed. I do still plan on running the device through a profile validator, and making intelligent decisions about failures/passes. |
|
|
|
Post by jautor on Oct 29, 2021 15:13:37 GMT
I'm also a bit lost with respect to the Redfish Profile Bundle, because it says there are supposed to be folders in the zip file, but I don't see folders. And the profiles in this bundle don't seems to validate using the Redfish-Interop-Validator against the Redfish-Profile-Simulator (not even a little bit, they are bugged or they're not actually profiles but schemas). The folders aren't there because there haven't been any profiles created internally by the DMTF, nor any submitted for republication at this point. We have been reluctant to create profiles ourselves, as the intent was for them to be driven by the consumers of Redfish data - to provide that "checklist" of what they need for their applications to do something useful. What happens when that is done from the service-side perspective is "here's all the data we've got"... You should not aim for 100% coverage in any of the schemas. There will always be properties your product doesn't support - and as a lot of them are based on underlying functionality - you would just be hard-coding a bunch of "disabled" properties that will give the users the wrong impressions. Jeff |
|
|
|
Post by jautor on Oct 29, 2021 15:17:05 GMT
That's interesting, mraineri , that you have some opinions about how the profiles could be used with respect to BMCs, because that's going to be my first and hopefully awesome Redfish service. I work for a systems manufacturer and we're building a new BMC to manage the new chassis with a computer system, backplane, and expansion-graphics.
I hope that found the right forum to continue more discussions about this implementing a Redfish service from scratch. From what I gather, after watching many hours of YouTube and reading as much as I can find, it seems as though when my boss says, "implement a Redfish service", that in truth I can sort of pick a depth and breadth to that as I see fit. I wonder if the old "implement what you have" is still a pretty decent way to proceed. I do still plan on running the device through a profile validator, and making intelligent decisions about failures/passes. Yes, "implement what you have" is still the right way to go. Your other target will likely be "whatever data you provide via other management interfaces (SNMP, etc.)" so that you can provide a migration path to get folks off of other, less standard (and secure) management protocols...
Jeff
|
|
