|
|
Post by wjmcnaughton on Nov 2, 2020 21:22:54 GMT
Has anyone had recent success generating client/server stubs from the openapi.yaml in the Redfish Schema Bundle? Should this be possible or am I doing something fundamentally wrong? If it's the latter, I apologize as this is all new territory for me. Any help would be much appreciated. I have followed this thread redfishforum.com/thread/342/openapi-generator-errors-redfish-schema but some of the links referenced appear to be out of date. Here is my attempt at code generate: openapi-generator-cli.jar generate -i openapi.yaml -g go -o ~/localtestdir This failed with numerous error similar to the following: "Failed to get the schema name: ./Task.v1_5_0.yaml#/components/schemas/Task_v1_5_0_Task" With the final error noting: Exception in thread "main" org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI). | Error count: 1, Warning count: 79 Errors: -Could not find definitions/context in contents of redfish.dmtf.org/schemas/v1/odata-v4.yaml Warnings: -Could not find definitions/context in contents of redfish.dmtf.org/schemas/v1/odata-v4.yamlI was able to generate client/server code by using the --skip-validate-spec option |
|
|
|
Post by mraineri on Nov 3, 2020 13:35:29 GMT
In the thread you've mentioned, we've came to the conclusion that given how many of the OpenAPI tools behave, we needed to republish our schema files to ensure all definitions are globally unique, and that everything we've published to date is largely not useful since folks have been unable to get code generation to work unless they perform various methods of scrubbing the schema files first. Unfortunately that means the openapi.yaml file from older schema bundles will be pointing to obsolete definitions due to the new naming methods for our objects. Could you try using the openapi.yaml file packaged with 2020.3?
|
|
|
|
Post by wjmcnaughton on Nov 3, 2020 14:34:32 GMT
Thanks for the quick response. I mistakenly copied the wrong link for the schema bundle into my post. I did download the latest version of the schema bundle (version 2020.3). Any suggestions on what needs to be scrubbed from the latest version to get code generation to work?
|
|
|
|
Post by mraineri on Nov 3, 2020 14:48:59 GMT
If everything is working as expected now, there shouldn't be anything needed to scrub at this point. There were some pending changes needed from the Swordfish group for everything to resolve, but as of a few days ago I've been able to run the OpenAPI spec validator against the 2020.3 version and everything resolves at this point. At least the errors regarding the OData context property were consistent to what I was seeing prior to the latest Swordfish files being posted.
Can you try performing the checks against an unmodified version of the 2020.3 files?
|
|
|
|
Post by wjmcnaughton on Nov 3, 2020 15:35:48 GMT
Extracted the Schema bundled and used it unmodified. I'm using 5.0.0-beta2 since I encountered the duplicate enum problem that was fixed with the addition of the "-p enumClassPrefix=true" option. java -jar openapi-generator-cli-5.0.0-beta2.jar generate -i ./openapi.yaml -g go-server -o ~/test-server -p enumClassPrefix=true --skip-validate-spec Here's a snippet of the start of the output: [main] WARN io.swagger.v3.parser.OpenAPIV3Parser - Exception while reading: java.lang.RuntimeException: Could not find definitions/context in contents of redfish.dmtf.org/schemas/v1/odata-v4.yaml at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:145) at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:56) at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefSchema(ExternalRefProcessor.java:918) at io.swagger.v3.parser.processors.ExternalRefProcessor.processSchema(ExternalRefProcessor.java:197) at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:220) at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:227) at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:154) at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:214) at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:37) at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:52) at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:67) at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:84) at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49) at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:67) at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16) at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:510) at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:560) at org.openapitools.codegen.cmd.Generate.execute(Generate.java:424) at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32) at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66) [main] WARN o.o.codegen.utils.ModelUtils - Failed to get the schema name: redfish.dmtf.org/schemas/v1/Message.v1_0_9.yaml#/components/schemas/Message_v1_0_9_Message |
|
|
|
Post by mraineri on Nov 3, 2020 15:46:24 GMT
Thanks! I'll start digging into this to see where that reference is coming from.
|
|
|
|
Post by wjmcnaughton on Nov 4, 2020 23:36:09 GMT
Any updates? Even though I'm able to generate code using openapi-generator, I don't believe it's correct. Taking ServiceRoot_v1_9_0_ServiceRoot as an example, it generates a structure that is populated with golang maps as shown below.
OdataContext map[string]interface{} `json:"@odata.context,omitempty"`
OdataEtag map[string]interface{} `json:"@odata.etag,omitempty"`
OdataId map[string]interface{} `json:"@odata.id"`
OdataType map[string]interface{} `json:"@odata.type"`
Is this due to the fact that it is unable to find the references for these fields as indicated in the errors from openapi-generator?
Could not find definitions/context in contents of redfish.dmtf.org/schemas/v1/odata-v4.yaml
|
|
|
|
Post by mraineri on Nov 5, 2020 16:11:24 GMT
I haven't been able to take a closer look yet.
As far as I know, that's how OpenAPI handles special characters in property names in GoLang; I believe that output is correct since it needs to convert the string "@odata.id" (and other OData properties) to something that is legal in the language (which would be OdataId).
Specifically for those messages with regards to the reference to odata-v4.yaml, I need to track down what schemas are referencing it incorrectly. All references should look something like this: "http://redfish.dmtf.org/schemas/v1/odata-v4.yaml#/components/schemas/odata-v4_context". The "definitions" notation is purely something that only exists in JSON Schema.
|
|
|
|
Post by wjmcnaughton on Nov 5, 2020 21:27:34 GMT
Those don't look like proper go field definitions for a JSON response. In go, a map[] requires a value/key pair. The field definition in for OdataId gets generated as a map when it should be a string. I suspect it's due to the fact that the proper odata-v4.yaml definition cannot be found. I was able to get openapi-generator to validate the openapi.yaml by: 1. pulling in the latest swordfish schemas: www.snia.org/sites/default/files/technical_work/PublicReview/swordfish/Swordfish_v1.2.1c_Schema.zip 2. converting all external references to dmtf & swordfish: sed -i *.yaml -e 's@http://redfish.dmtf.org/schemas/v1/@./@ sed -i *.yaml -e 's@http://redfish.dmtf.org/schemas/swordfish/v1/@./@' 3. the swordfish version of VolumeCollections has invalid $ref paths. Had to change all references of "$ref: ./odata-v4.yaml#/definitions/context" to "$ref: ./odata-v4.yaml#/components/schemas/odata-v4_context" I can now successfully generate code and have a framework for the Redfish API server. The ServiceRoot_v1_9_0_ServiceRoot struct now has the proper definition for the OdataId field: OdataId string `json:"@odata.id"` There is an error in openapi.yaml where the /redfish/v1 path is defined twice. This causes problems in the generated go code. After removing the duplicate, the generated code appears to be working properly. |
|
|
|
Post by mraineri on Nov 6, 2020 13:43:14 GMT
Thanks. I'm looking at VolumeCollection from Swordfish and I'm not seeing any of the OData references mentioning "/definition/context"; the @odata.context reference for me is showing "$ref: redfish.dmtf.org/schemas/v1/odata-v4.yaml#/components/schemas/odata-v4_context". You're right about what the output is supposed to be for the OData properties; the important portion is the "json: @odata.id" text at the end, so "map" isn't needed. I'm not seeing where "/redfish/v1" is defined twice. We have both "/redfish/v1" and "/redfish/v1/" (one with the trailing slash and one without) since they are both mandatory in the Redfish Specification. Does server-side code do anything unique when it comes to trailing slashes in URIs? I'm still looking at why the references are failing unless you change everything to reference a local file. Following the reference by entering the URI in a browser works fine for me. Turning up the verbosity of openapi-generator isn't useful for this it seems, so I'll probably need to build a debug version with more info. |
|
|
|
Post by wjmcnaughton on Nov 6, 2020 16:43:08 GMT
We inadvertently copied an outdated version of VolumeCollection.yaml from redfish.dmtf.org/schemas/swordfish/v1/DriveCollection.yaml. That explains the references to /definition/context. It looks like we needed some updates from swordfish that aren't in the current DSP8010_2020.3 Schema bundle? I missed that minor distinction between the "/redfish/v1" and "/redfish/v1/" paths. However, having both results in errors in the generated go code: go/api_default.go:3463:5: c.RedfishV1Get_0 undefined (type *DefaultApiController has no field or method RedfishV1Get_0) go/api_default.go:20328:26: c.service.RedfishV1Get_1 undefined (type DefaultApiServicer has no field or method RedfishV1Get_1) go/api_default_service.go:25:9: cannot use &DefaultApiService literal (type *DefaultApiService) as type DefaultApiServicer in return argument: *DefaultApiService does not implement DefaultApiServicer (missing RedfishV1Get_0 method) The problem seems to be that the DefaultApiController defines Routes for: RedfishV1Get & RedfishV1Get_0 but all of the function definitions are for: RedfishV1Get() and RedfishV1Get_1(). Changing all references of RedfishV1Get_1 -> RedfishV1Get_0 fixes this. |
|
|
|
Post by mraineri on Nov 6, 2020 19:44:06 GMT
DSP8010 should be standalone, so if you don't have any need for Swordfish extensions, you don't need to download the Swordfish bundle. However, if you're working on a Swordfish service, then you'll need both. DriveCollection is in the 8010 bundle as of 2020.3 since it was contributed by SNIA to the DMTF.
We'll need to discuss having both "/redfish/v1" and "/redfish/v1/" in the openapi.yaml file a bit. Originally we just listed one of them, but it was pointed out to us that since this could be used to generate service-side code, both need to be called out since both are currently required. Maybe there's some better way of augmenting the definitions in the openapi.yaml file to avoid that sort of conflict you're seeing. I'll add that to the list of things to look into more.
|
|
|
|
Post by mraineri on Nov 12, 2020 16:10:59 GMT
Just an update on this... Still digging into what's going on, and I'm off trying to trace how remote files are fetched, verified, etc. I've been trying to mimic what the Swagger Parser library is doing in terms of HTTP request headers and seeing if the responses come back correctly and so far everything looks correct. Still not sure where things are going wrong.
|
|
|
|
Post by wjmcnaughton on Nov 24, 2020 18:18:17 GMT
I appreciate you following up on this issue. I have been able to make progress on a Redfish API service by converting all the external references to local ones.
We tried generating client libraries and ended up with an issue related to ./Resource.v1_0_0.yaml#/components/schemas/Resource_v1_0_0_Resource in Resource.yaml. The code generator creates the following go structure which makes go very unhappy:
type ResourceResource struct {
interface{} *interface{}
}
Have you had any success with go client libraries generated using openapi-generator? For the time being, we are using a ready made redfish client (go-redfish)
|
|
|
|
Post by mraineri on Dec 3, 2020 17:59:58 GMT
I know some folks have had varying degrees of success, but after modifying the files. I know Go clients have been built in this manner.
I've been trying to understand the changes others have been making in order to incorporate them into the published files. But specifically for that new issue you've found with ResourceResource, I have not seen that reported before and will need to dig deeper.
|
|
