4
$\begingroup$

While on some SEs, notably Stack Overflow, it is often sufficient for questions to contain a short code snippet, others, notable TeX/LaTeX SE, ask for a full Minimal Working Example (MWE).

I'm bringing this up because I saw this question yesterday. I think it's a good question, clear and with a screenshot, very googlable and hopefully very relevant to future visitors. It even includes the shader code which is nice. However, unless someone spots an obvious problem with that handful of lines of shader, it is really hard for answerers to actually debug the issue. They'd need to set up some OpenGL boilerplate, as well as a scene to try and see if they can reproduce it. That's quite a lot to ask from people who spend their freetime helping others. And even if they did, the issue might ultimately not be in the shader and so the different boilerplate code might not exhibit the same problem.

If the post instead contained the simplest complete program to exhibit the problem (be it C, C++, Python or JavaScript+WebGL), anyone could just get the programming running and play around with it.

So my question is, what should our policy on this be? Should all questions which are about specific code include a full MWE? If so, how do we point (especially new) users to what exactly is required for an answerable question?

(PS: If we do want all specific-code questions to contain MWEs, Stack Snippets would be even more useful.)

$\endgroup$
  • $\begingroup$ That might be hard given that the minimal "hallo triangle" code for vulkan is claimed to be 400+ lines. $\endgroup$ – ratchet freak Aug 31 '15 at 10:25
  • 1
    $\begingroup$ @ratchetfreak I think that's exactly why it is important. There is a lot more context to graphics code than just the shader, and likely no two people will use the exact same 400 lines of boilerplate code, which could be significant. If you've got a problem that really is just a shader, your MWE could be a Shadertoy or similar. But in general, when asking a question about specific code someone will have to narrow down the problem by building an MWE, and I think that shouldn't be the answerer's responsibility. $\endgroup$ – Martin Ender Aug 31 '15 at 10:29
  • $\begingroup$ Although the code to demonstrate the problem could be long, the asker presumably already has this code or they wouldn't know they had the problem. So this would only be extra work for the asker if the rest of the code is something they are not permitted to post here for some reason (incompatible license or secret project). $\endgroup$ – trichoplax Aug 31 '15 at 11:09
  • 1
    $\begingroup$ @trichoplax Well the focus is on "minimal". So ideally the asker shouldn't just dump their entire code into the question, but try to strip it down as far as possible before posting (you're still right of course: stripping down existing code should be easier than trying to find the minimal boilerplate necessary for an isolated piece of shader code). $\endgroup$ – Martin Ender Aug 31 '15 at 11:11
  • $\begingroup$ Yes - I agree that should also be the asker's job. I'm trying to piece together an answer and I guess the main problem here is deciding what counts as minimal. $\endgroup$ – trichoplax Aug 31 '15 at 11:13
2
$\begingroup$

Not every question here will be a code question, but for those that are, clearly a minimal working example is better, so we need to decide how to respond to questions that have code that is longer than necessary, or shorter than necessary. Both cases are a problem: Too much code is more work for our experts (not just taking advantage of their time but also depriving other questions of their attention). Too little code means the problem no longer occurs and so also wastes time.

If we comment to indicate what needs to change, and then leave it to the asker, some askers will ignore the comment and never touch their question again, leaving us with an unanswered question to forever clutter our question list. The same goes for downvoting questions that aren't ready to be answered (and I don't like the idea of downvoting well-meaning questions).

I think the answer is to vote to close such questions. This puts them on hold until they are ready to be answered, so other questions don't suffer from loss of attention. Once the code is ready for experts to look at, the question can be reopened and answered much more efficiently. If we can emphasise that this is not a bad thing, just a natural part of the life cycle of a question to ensure the best answers, then hopefully everyone will see voting to close as positive and we can get such questions closed quickly, dealt with, and then reopened quickly. This also solves the problem of askers who never come back - their questions will remain closed unless someone else chooses to edit the question to rescue it.

The decision on where to draw the line will be a subjective one. I wouldn't want to close a question for having one punctuation mark more than strictly necessary to reproduce the problem. But I think we should have a policy of closing questions that are in need of a Minimum Working Example, and then the votes of the community will decide where the line is on a case by case basis.

Since the problem questions will often be from new users who are not yet familiar with our site, let's aim to give comments that explain why the question is being put on hold (sounds less final than "closed"), and mention that it can be reopened once it has been improved.

$\endgroup$
  • $\begingroup$ As discussed in chat, we might call them MCVEs instead and maybe ask a dev to add this help centre page to CG.SE for reference. I'll also have a read around meta.tex.SE to see what their guidelines are. $\endgroup$ – Martin Ender Aug 31 '15 at 11:56
  • $\begingroup$ Browsing through the TeX meta post there's also short, self-contained, correct example (SSCCE) which sounds like it is more commonly used in programming contexts (with MWE used in the TeX community). I'm particularly fond of the V for Verifiable in MCVEs though. $\endgroup$ – trichoplax Aug 31 '15 at 12:02
0
$\begingroup$

I think we should only ask for a Minimal Working Example, if it is necessary. Especially in our context, drawing and shading a triangle using the favorite API can already be a huge amount of code. Huge in the context of our Q&A site. Moreover, in most cases it will be completely superfluous to have some code for e.g. the Main (Game) Loop and all the API setup stuff in the question.

And even if it actually is there and is actually truncated to the tiniest amount of code necessary to compile and run and experience the problem: You'll soon see that you can't compile it, because e.g. you're using OS X instead of Windows and thus using some other compiler, have different headers and some things are simply not available on all systems, because everyone likes to define his own "standard".

This might work for (La)TeX, because as far as I have seen there are no big dependencies on the environment and the MWE is probably quite short in most cases. But in CG context this would simply be too much code.

If you have a question about a shader, then in most cases shader code will be sufficient. If you have a question about weirdly misplaced objects, then parts of your scene graph will probably be enough. Everything unrelated to the problem will only distract from the problem.


Edit, just to make this clear: Yes, the code should be stripped down to the minimum. But only ask for a copy/paste/compile/run-example (MWE), if necessary.

$\endgroup$
  • 1
    $\begingroup$ I think this is a reason to expect people to mention their operating system and environment in the question. Then people who don't have access to that set up can move on to other questions. $\endgroup$ – trichoplax Aug 31 '15 at 16:45
  • 1
    $\begingroup$ "in most cases it will be completely superfluous to have some code for e.g. the Main (Game) Loop and all the API setup stuff in the question." Anything that's superfluous is not part of the MWE. "But in CG context this would simply be too much code." The problem is that as an answerer I need that code, otherwise I won't be able to test my own answer before posting it and can't ensure that it's actually working and therefore a high-quality answer. So if the OP doesn't provide the MWE, the only alternative is for the answerer to write it themselves which shifts the work to the wrong person. $\endgroup$ – Martin Ender Aug 31 '15 at 17:30
  • $\begingroup$ Of course, there will be cases where this might not be necessary because someone can spot an obvious flaw, but I think more often than not it could be more tricky than that. And for those cases it would be nice if we just made it a good community habit to strip problems down to a bare minimum. $\endgroup$ – Martin Ender Aug 31 '15 at 17:31
  • $\begingroup$ @MartinBüttner Yes, of course the problem should be stripped down. But often the bare minimum that's needed is far less than a bunch of lines of code that can be copy-pasted to a file, compiled and run. If my Scene Graph is broken, I'd post code of my Scene Graph. Still, to use it, you need a working MainLoop, API setup, load objects, mouse or touch interaction (or default movement), etc... Now I provide this in Swift ready to compile and run on iPad. Maybe you're the Scene Graph expert, but you neither have a Mac nor iPad. And now? You have lots of distracting code, you can't even use. $\endgroup$ – Nero Aug 31 '15 at 21:25
  • $\begingroup$ And yes, I see the advantage of having a compilable and runnable example. But this is not TeX, in which code will (maybe/probably) compile on every platform. Here we have a wide variety of languages and you can't assume that the one, who likes to help you, uses the same environment or languages. Nevertheless, an experienced programmer will be able to read and understand code in languages he/she (currently) does not or can not use. $\endgroup$ – Nero Aug 31 '15 at 22:08
0
$\begingroup$

I broadly agree with Nero on this, but I'd like to clarify one point. I don't think it's necessarily clear at the outset when a question is "about specific code" and when it's a matter of general understanding. And ratchet freak is correct that with next-generation graphics APIs, a "minimal example" may not be very minimal at all.

A MWE is the gold standard for a code-based question, no dispute there. This is especially true with something like APIs which are basically complex state machines. However OpenGL, even more useful than seeing the code would be seeing the state of the machine right at the point where an API call did something that you didn't expect. That's more useful than tracing through code, IMO.

Besides, what if the MWE includes multiple kilobytes of model data?

So here's my suggestion:

  • At a minimum, you need to include enough code (and screenshots) so that potential answerers can understand the problem.
  • Questioners should expect that more information (including more code) may be asked for, and that there may be no answer without a MWE.
  • For OpenGL (and anything else relevant), we may want to write some code which dumps the state, put the code somewhere, and then ask people to run that at various points in the code and post the dump. Something like OpenGLDebugFrameDump from UE4 if you've seen that.
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .