After the last call of the process cg, my understanding of the status of this pull request is that:

1. we generally agree on the second part of the pull request (starting at https://github.com/w3c/w3process/pull/214/files#diff-eacf331f0ffc35d4b482f1d15a887d3bR1966)
2. We want to quibble some more about the first part:
   • @nigelmegitt wants to make sure that we don't disallow a WG intentionally leaving things undefined, and only disallow intentionally leaving mistakes and bugs
   • @dwsinger wants to make sure that we require the text to be at the level of quality expected from a REC, not (if I understand correctly) in the sense of technical correctness (as that's sufficiently covered), but in the sense of being quality prose, that isn't an incomprehensible mess, or full of grammar and spelling errors.

I'd rather not lose the sense of "Candidate Recommendations are expected to be acceptable as Recommendations", and I wonder if we should have "The text of a Candidate Recommendation is expected to be of the quality of, and acceptable as, the text of a Recommendation itself." or the like?

I understand what you're trying to achieve with that phrasing, but I am not sure that it works.

• We do not actually have a requirement that the prose of RECs be high quality prose, separately from the technical requirements. I don't know how to require than other than observing after wide review + an AC vote that nobody is complaining that the spec is crap. And I think we already have that. So maybe we don't need anything.
• Saying it's about "the text" sounds it might exclude non textual things like figures, which is certainly not what you intend.

Assuming you don't want to just drop this, how about that:

A Candidate Recommendation is expected to be of the quality of a Recommendation itself, and acceptable as such if and when the requirements for further advancement are met.

"The text of a Candidate Recommendation is expected to be of the quality of, and acceptable as, the text of a Recommendation itself."

@dwsinger the test for this is that it passes the Director's, and the AC's review, right, because those are the reviews that happen to get from CR to PR and then Rec? The Director reviews the transition to CR, but the AC isn't likely to comment until PR. We either need to close that loop so that your goal can be verified before transitioning to CR, or open up the very real possibility of getting feedback along the lines "this should never have been a CR" after the spec is in CR and has already met its CR exit criteria. The process needs to allow for the right feedback to happen at the right time, or accept that such feedback cannot be meaningfully processed. What would you do here?

To get accepted as a CR, the spec needs not only to pass the Director's personal review, but also to demonstrate wide review. That seems like a perfectly appropriate time for anyone to point out if a spec is such a mess that it shouldn't be allowed to go forward.

@frivoal right, so in that case does adding this statement of expectation help anyone? It seems like it's more an instruction to reviewers on the level of quality they should be aiming for, rather than a part of the process per se.

I think we need, at least, to set an expectation, that the team and others can point at: if, magically, implementation experience and interop and all the things that happen post-CR go amazingly swimmingly, you should basically have the PR/Rec. text in hand. The spec needs to be of specification-quality, basically, or as close as it can get without the CR-period experience. And we need to say that that is the expectation.

@frivoal right, so in that case does adding this statement of expectation help anyone? It seems like it's more an instruction to reviewers on the level of quality they should be aiming for, rather than a part of the process per se.

In a sense, yes. reviewers should be able to say "nope, your document is not spec.-ready and it's required, and you knew it was required because it's written in the process". I think if it's unstated we'll get cases trying to slip by, and slip by they will.

@dwsinger your comments seem to be in support of the phrasing I proposed in #214 (comment), but you don't say so explicitly. Does that work for you, or should we try some more?

@dwsinger , @nigelmegitt I've updated the PR with a blend of the phrases we have been discussing. I hope the latest version covers your expectations.

To make it easier to read than looking at a diff, here are the two key paragraphs (do look at the diff if you want to see them in context)

Note: Advancing to Candidate Recommendation indicates either that no further improvement is expected without additional implementation experience or feedback from the wider community. A Candidate Recommendation is expected to be of the quality of a Recommendation itself, and acceptable as such if and when the requirements for further advancement are met. Announcement of a different next step should include the reasons why the change in expectations comes at so late a stage.

The decision to advance to a new maturity level must be made solely based on the technical maturity of the work.
Note: should faster advancement to meet scheduling considerations be desired, this may be achieved by reducing the scope of the technical report to a subset that is adequately mature and deferring less stable features to other technical reports.

This is fine. Thanks for persisting

@dwsinger Should I just merge? Do you want to issue a CfC?

Looks good to me after the grammar tweak I've proposed is made.

Done.

Wrt

A Candidate Recommendation is expected to be of the quality of a Recommendation itself,
and acceptable as such if and when the requirements for further advancement are met.

I really liked @dwsinger’s phrasing in this comment:

The point of the CR transition is to ask for implementations, implementer feedback, testing, and so on. In the miraculous case that this all goes 100% perfectly, the text of the document should be good enough to publish as a Recommendation. That's what I think that the Process requires, and should be required. I think it irresponsible to ask people to implement and give feedback on something that is not well enough written, consistent, or technically complete enough to publish (and hence, not enough to implement thoroughly either). So, I think the sentence should stand.

So maybe something like “A Candidate Recommendation is expected to be as well-written, detailed, self-consistent, and technically complete as a Recommendation, and acceptable as such ...”?

@dwsinger commented a while back

reviewers should be able to say "nope, your document is not spec.-ready and it's required, and you knew it was required because it's written in the process". I think if it's unstated we'll get cases trying to slip by, and slip by they will.

This suggests a pandora's box of subjectiveness that I'm loathe to open in the text of the Process. By the time a WG feels their spec is sufficiently mature to become a CR all reviewers with concerns should be expected to propose -- or at least outline -- changes that would resolve their concern. "Not spec-ready" is only a useful comment when accompanied by sufficient text for the WG to understand what changes are being asked of it. 6.2.2 and 6.4 hold the normative prose for reviewers to make such comments and have them considered during transition deliberations.

6.1.2 is the wrong place to be providing advice to Working Groups, their editors, and reviewers.

I don't support the proposed additions to 6.1.2; I only support dropping the Note as an historic artifact as it apparently now causes confusion. Additional informative advice such as that proposed could usefully be incorporated into Working Group Effectiveness.

Removing the note altogether was one of the proposals I suggested in #172, so it would satisfy the issue as far as I'm concerned. I actually think the modification as proposed in this pull request is better than that, but complete removal would also be acceptable to me.

This suggests a pandora's box of subjectiveness that I'm loathe to open in the text of the Process. By the time a WG feels their spec is sufficiently mature to become a CR all reviewers with concerns should be expected to propose -- or at least outline -- changes that would resolve their concern.

Agreed. A claim that something is not spec.-ready would need to be supported by evidence (for example, the existence of a TBD in the document, or editorial notes indicating uncertainty, and so on).

"Not spec-ready" is only a useful comment when accompanied by sufficient text for the WG to understand what changes are being asked of it. 6.2.2 and 6.4 hold the normative prose for reviewers to make such comments and have them considered during transition deliberations.

If you want to move the requirements and advice to 6.4, which is also about Candidate Recommendation, I am fine with that.

I don't support the proposed additions to 6.1.2; I only support dropping the Note as an historic artifact as it apparently now causes confusion. Additional informative advice such as that proposed could usefully be incorporated into Working Group Effectiveness.

So, you think that we should return to the old days where groups issued Last Calls (now CRs) on documents that could not be considered as publishable? I don't, and I don't understand why you would.

If you want to move the requirements and advice to 6.4, which is also about Candidate Recommendation, I am fine with that.

I want any new proposed normative requirements to be in 6.4 and I recommend that advice be in a separate document. Statements such as "The decision to advance to a new maturity level must be made solely based on the technical maturity of the work" are claimed in this thread to not be constraining but have the same type of Process consistency issue that opened #172.

So, you think that we should return to the old days where groups issued Last Calls (now CRs) on documents that could not be considered as publishable?

No, certainly not.

@swickr OK, then retain:

• "A Candidate Recommendation is expected to be as well-written, detailed, self-consistent, and technically complete as a Recommendation, and acceptable as such if and when the requirements for further advancement are met."
• "The decision to advance to a new maturity level must be made solely based on the technical maturity of the work."

Move to guide:

• "Note: Should faster advancement to meet scheduling considerations be desired, this can be achieved by reducing the scope of the technical report to a subset that is adequately mature and deferring less stable features to other technical reports." is advice that states the somewhat-obvious and could be in the guide.

Omit or move to guide:

• "Note: Advancing to Candidate Recommendation indicates that no further improvement is expected without additional implementation experience and testing." could be omitted as it's the same as the statement of expectation.

The retentions could be in 6.4, which is about CR.

Is this your proposal? If not, please be specific.

(Honestly, we're supposed to be in yes/no decision now; the time for this level of tinkering is supposed to be past.)

"The decision to advance to a new maturity level must be made solely based on the technical maturity of the work."

is over-constraining as @wseltzer previously objected.

Section 6.1.2 is the definition of maturity levels, not advice on how to reach them. I propose a different approach to remain definitional while respecting the spirit of the pull request:

To the end of the definition of Working Draft add

The content of a Working Draft is not required to be stable or to describe a stable design.

In the definition of Candidate Recommendation replace

Note: Candidate Recommendations are expected to be acceptable as Recommendations. Announcement of a different next step should include the reasons why the change in expectations comes at so late a stage.

with

The content of a Candidate Recommendation is expected to be stable and of sufficient quality and maturity to proceed eventually to Proposed Recommendation though implementation experience may result in adjustments, including removal of features.

and move the remainder to a guide.

I would be OK with "well-written, detailed, self-consistent, and technically complete" in place of "of sufficient quality and maturity"

"The decision to advance to a new maturity level must be made solely based on the technical maturity of the work."

is over-constraining as @wseltzer previously objected.

You seem to understand something from Wendy here that no-one else does, so can you explain? You (plurally) want to be able to advance technically immature work?

No, the decision to -- or not to -- advance may be made on other considerations. For example, the group may decide not to advance work that is technically mature, in order to synchronize it with other work that is not yet mature.

"Only sufficiently technically mature work should be advanced," is different in meaning from "The decision to advance to a new maturity level must be made solely based on the technical maturity of the work." I'd find the first acceptable, not the second.

The original statement says nothing at all about why you make a decision not to advance, only about the criteria to advance. Nonetheless, if you can re-phrase to something which has the same intended meaning, I expect to get consensus.

@dwsinger clearly our interpretations of the phrase and what's said by implication differ. I suggest "Only sufficiently technically mature work should be advanced." Otherwise, I suggest we leave it out entirely.

try again.

@swickr @wseltzer OK,

retain:

"A Candidate Recommendation is expected to be as well-written, detailed, self-consistent, and technically complete as a Recommendation, and acceptable as such if and when the requirements for further advancement are met."

replace
"The decision to advance to a new maturity level must be made solely based on the technical maturity of the work."
with
"Only sufficiently technically mature work should be advanced."

Move to guide:

"Note: Should faster advancement to meet scheduling considerations be desired, this can be achieved by reducing the scope of the technical report to a subset that is adequately mature and deferring less stable features to other technical reports." is advice that states the somewhat-obvious and could be in the guide.

Omit or move to guide, or leave as note with the retentions:

"Note: Advancing to Candidate Recommendation indicates that no further improvement is expected without additional implementation experience and testing."

The retentions could be in 6.4, which is about CR, or left in the section that the original phrase is in.

@dwsinger 's proposal in #214 (comment) is acceptable to me. I'd prefer keeping the notes in instead of moving then the the guide or dropping, as I think such notes improve the understandability of a document more than terseness does, but if there is an actual objection to having them, and no objection if they're removed, I can live with that.

+1 @dwsinger's #214 (comment)

replace
"The decision to advance to a new maturity level must be made solely based on the technical maturity of the work."
with
"Only sufficiently technically mature work should be advanced."

I have updated the pull request with this change, since it seems that it is at the core of Wendy's objection and essential to Ralph's, and is non controversial for those who already supported this PR, as the rephrasing continues to match what we were attempting to say.

As for moving the text to another section and dropping (in favor of the guide) one or both of the notes, I'm awaiting further instructions from the chair. It seems to me that these changes are not essential to overcome the objection from Wendy and Ralph, and that other people were fine with the text as it is, so I'd rather not change it at this stage, but I have no objection to doing so if the chair judges that doing so is the best way to get to consensus.

The rephrasing addresses my core objection and with that change I accede to a consensus for including this advisory material in the Process rather than in /Guide.

Looks good to me, too.

The CfC about the 2019 process is closed and the chair has concluded we do have consensus. Merging.