|
|
Post by Caffrey on Apr 13, 2022 2:18:33 GMT
Hi, I would like to ask for suggestion about tools released on github.com/DMTF. The situation we are facing now is many of our customers always require us to run most tools on DMTF github so I would really need your help to see if it's possible to provide a tool list that all Redfish developers shall run on their product as quality assurance. Most of our customers treat the reports / logs generated by DMTF tool as official certification. Many thanks.  BR, Caffrey |
|
|
|
Post by jautor on Apr 13, 2022 21:30:27 GMT
The two conformance tools that every developer / QA team should run against their implementation are: Redfish Service Validator - github.com/DMTF/Redfish-Service-ValidatorThis tool is focused on the JSON payload and schema conformance. Start your testing here as data/payload errors are more likely to cause interoperability issues for end users. This tool uses the Redfish Python library, so "critical" protocol issues may prevent it from running to completion. If so, swap the order of your testing. Redfish Protocol Validator - github.com/DMTF/Redfish-Protocol-ValidatorThis tool is focused on the HTTP protocol conformance, checking headers, response codes and other protocol functions. While many of the issues it detects may not directly impact end users or simple scripts, they will contribute to interoperability issues with more sophisticated client software and automation tools. Both of these tools are actively maintained by DMTF personnel. If you encounter issues with either tool. please open issues on GitHub. Note that both tools have been run on numerous implementations, so anything that crashes the tool very likely indicates an issue with the implementation (as well as something we should catch). We may ask that you attach the log file created with the "--debugging" option. Pull requests for enhancements or bug fixes are also welcome! In addition, developers can utilize the "Use Case Checkers" to evaluate the implementation of specific functional portions of the schema/protocol. github.com/DMTF/Redfish-Usecase-CheckersThanks for the great question - it also reminded me that we should add some developer/debugging recommendations to the GitHub readme text as well. Jeff |
|
|
|
Post by Caffrey on Apr 14, 2022 2:20:15 GMT
Glad to help! Thank you so much for the valuable recommendation. Caffrey |
|
|
|
Post by AMI_Mani on Aug 7, 2023 12:43:01 GMT
Hi Jeff, Thanks for listing conformance tools to use Redfish Service Validator, Redfish Protocol Validator. When I checked Redfish Conformance Test Suite Guidance document in below link www.dmtf.org/sites/default/files/standards/documents/DSP-IS0018_1.0.0.pdfIt mentioned about Redfish Interop Validator also and it depends on profile created by vendor systems or system tools based on a vendor's specification in a profile. Do we have any profile available from DMTF Redfish to check against implementation(generic profile based on Redfish schema published by DMTF)? If not can we change this tool as optional or recommend like Redfish Interop Validator v1.1.7 with the appropriate Redfish profile(Recommended) (like Redfish Usecase Checker v1.0.6 (recommended)) in DSP-IS0018_1.0.0.pdf?Thanks, Mani |
|
|
|
Post by mraineri on Aug 7, 2023 14:20:04 GMT
DMTF does not produce profiles; we might republish them at the request of another organization though. Keep in mind profiles are much more specific to expectations from a given type of product; there's really no "generic Redfish" profile. This all comes down to 1) what it is you're building and 2) what functionality your users are expecting out of the interface. OCP does produce their own profiles for their own manageability requirements for different types of products. You can find them here: github.com/opencomputeproject/HWMgmt-OCP-Profiles |
|
|
|
Post by AMI_Mani on Aug 7, 2023 17:21:33 GMT
Thanks for details.
OCP profiles available in github.com/opencomputeproject/HWMgmt-OCP-Profiles is old version compared to new schema supported by DMTF, beacuse of this only checked for any plan of DMTF to publish profiles based on latest schema
Thanks,
Mani
|
|
|
|
Post by mraineri on Aug 7, 2023 18:13:43 GMT
Again, profiles convey a set of user requirements for a particular environment. So, if OCP has requirements around particular properties, they will be listed in their profile. Simply "using the latest schema" is not encouraged since it doesn't explain what a user is trying to do. Users write their management software around particular properties in responses, so profiles are structured as such; profiles contain the requirements for their software to successfully integrate Redfish services into their environment. If OCP's management requirements do not change, I would not expect profiles to change either. However, OCP is certainly open to taking feedback to add or update their profiles and/or requirements, but DMTF is currently not looking to define product requirements, so you will not see DMTF-authored profiles at this time.
|
|
