Why does this JNI call segfault on Windows only?

Question

I have some C code that adds strings to a Java array using JNI. The call to NewStringUTF segfaults -- but only on Windows 7 32-bit (in a VirtualBox VM, which is all I have to test on). In some cases, it makes it to SetObjectArrayElement call and then segfaults.

void launch_jvm_in_proc(mrb_state *mrb, CreateJavaVM_t *createJavaVM, const char *java_main_class, const char **java_opts, int java_optsc, const char **v, int prgm_optsc) {
  int i;
  JavaVM *jvm;
  JNIEnv *env;
  //...
  jclass j_class_string = (*env)->FindClass(env, "java/lang/String");
  jstring j_string_arg = (*env)->NewStringUTF(env, "");
  jobjectArray main_args = (*env)->NewObjectArray(env, prgm_optsc, j_class_string, j_string_arg);

  for (i = 0; i < prgm_optsc; i++) {
    j_string_arg = (*env)->NewStringUTF(env, (char *) prgm_opts[i]);
    if (!j_string_arg) {
        mrb_raise(mrb, E_ARGUMENT_ERROR, "NewStringUTF() failed");
    }
    (*env)->SetObjectArrayElement(env, main_args, i, j_string_arg);
  }
  //...
}

There are also cases where the call is made to SetObjectArrayElement successfully, and then it consistently fails on the third iteration of the loop (when i=2). This happens when I consume this project a library in mjruby. I can't explain that either.

The complete project is on Github in mruby-jvm.

Error Details:

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name: mruby-jvm.exe
  Application Version:  0.0.0.0
  Application Timestamp:    55eb01a5
  Fault Module Name:    mruby-jvm.exe
  Fault Module Version: 0.0.0.0
  Fault Module Timestamp:   55eb01a5
  Exception Code:   c0000005
  Exception Offset: 0003fff2
  OS Version:   6.1.7601.2.1.0.256.4
  Locale ID:    1033
  Additional Information 1: 0a9e
  Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
  Additional Information 3: 0a9e
  Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

Is there a way to collect more information on the error?

It works perfectly on Linux and Mac.

I've included instructions on how to reproduce the problem in this Github issue.

EDIT

I should clarify that I've examined this every way I can. I've checked that the various args are not NULL. I even get condensed almost the entire program to this:

static void
jvm_wtf(const char *java_dl, const char *jli_dl) {
  JavaVM *jvm;
  JNIEnv *env;
  JavaVMInitArgs jvm_init_args;
  CreateJavaVM_t* createJavaVM = NULL;

  jvm_init_args.nOptions = 0;
  jvm_init_args.version = JNI_VERSION_1_4;
  jvm_init_args.ignoreUnrecognized = JNI_FALSE;

#if defined(_WIN32) || defined(_WIN64)
  disable_folder_virtualization(GetCurrentProcess());
  HMODULE jvmdll = LoadLibrary(java_dl);
  createJavaVM = (CreateJavaVM_t*) GetProcAddress(jvmdll, "JNI_CreateJavaVM");
#elif defined(__APPLE__)
  // jli needs to be loaded on OSX because otherwise the OS tries to run the system Java
  void *libjli = dlopen(jli_dl, RTLD_NOW + RTLD_GLOBAL);
  void *libjvm = dlopen(java_dl, RTLD_NOW + RTLD_GLOBAL);
  createJavaVM = (CreateJavaVM_t*) dlsym(libjvm, "JNI_CreateJavaVM");
#else
  void *libjvm = dlopen(java_dl, RTLD_NOW + RTLD_GLOBAL);
  createJavaVM = (CreateJavaVM_t*) dlsym(libjvm, "JNI_CreateJavaVM");
#endif
printf("Begining\n");
  createJavaVM(&jvm, (void**)&env, &jvm_init_args);
  jclass main_class = (*env)->FindClass(env, "Main");
  jmethodID main_method = (*env)->GetStaticMethodID(env, main_class, "main", "([Ljava/lang/String;)V");
  jclass j_class_string = (*env)->FindClass(env, "java/lang/String");
  jstring j_string_arg = (*env)->NewStringUTF(env, "");
printf("Checking for NULL\n");
  if (!createJavaVM) { printf("createJavaVM is NULL\n");}
  if (!main_class) { printf("main_class is NULL\n");}
  if (!main_method) { printf("main_method is NULL\n");}
  if (!j_class_string) { printf("j_class_string is NULL\n");}
  if (!j_string_arg) { printf("j_string_arg is NULL\n");}
printf("Right before segfault\n");
  jobjectArray main_args = (*env)->NewObjectArray(env, 1, j_class_string, j_string_arg);
printf("It won't get here\n");
  (*env)->SetObjectArrayElement(env, main_args, 0, (*env)->NewStringUTF(env, "1"));
  (*env)->CallStaticVoidMethod(env, main_class, main_method, main_args);
}

Now I get a segfault at NewObjectArray. Some googling has led me to believe that this may result from Windows terminating the program because it thinks the memory allocation by the JVM is malicious. How would I determine if this is true?

You haven't really given us enough to give a sure answer. Supposing that we can discount the possibility of a bug in the VM, the basic answer is that something about your C code invokes undefined behavior. Based on what you presented, that more likely than not involves some sort of data malformation. I'd particularly scrutinize array `prgm_opts`, which is proximal to the issue. Its provenance, type, and contents are not evident from the code you presented, so there's nothing more I can say about it. — John Bollinger, Sep 05 '15 at 15:31
Unfortunately, I don't think a sure answer is possible without actually checking out the code and testing -- but i'm not expecting that to happen. I am hoping that there is some common mistake or other fundamental error in how this code is set up, and maybe someone who has done this kind of thing before might recognized it. I don't think that's an unreasonable question. — codefinger, Sep 05 '15 at 17:30
"It works perfectly on Linux and Mac." That seems unlikely -- or else trivial -- given that as now presented, the code is *unused* on Linux and Mac. In any event, the nature of undefined behavior is that you cannot rely on it. If your code has undefined behavior then it is entirely possible that it reliably works as you expect on one system, and reliably fails on another. — John Bollinger, Sep 05 '15 at 23:46
Your usage of the JNI invocation API is a little odd. One would normally expect the JNI code to be linked directly to the JNI library, so that you don't have to load the library manually. It is conceivable that the library doesn't like being loaded that way. If you are right that Windows thinks the allocation is malicious, then it is also conceivable that the allocation being performed by a manually-loaded library contributes to Windows' evaluation. — John Bollinger, Sep 05 '15 at 23:54
Note, however, that Windows certainly is *not* terminating the program, at least not in a controlled manner such as I would expect for a security control coming into play. A segfault is anything but that. If you want to check further on that, though, then look at the system logs. — John Bollinger, Sep 06 '15 at 00:01
This declaration is suspicious: `JavaVMOption jvm_opts[0];`. Zero-length arrays are not allowed in C. Your compiler *ought* to reject the declaration. If for some reason it does not, then the program's behavior is undefined. The fact that you set `jvm_init_args.nOptions = 0` might rescue this in practice, but undefined is undefined -- there cannot be a *reliable* way to rescue it other than to remove or fix the code exhibiting UB; otherwise the behavior would be defined. — John Bollinger, Sep 06 '15 at 00:17
I've been over your code -- both versions presented here, and the actual full affected function on GitHub. It looks pretty solid except for the problem with zero-length arrays. In the real code, `jvm_opts` is a variable-length array, so the problem case of zero-length is less obvious. I don't know whether this is really the cause of your segfault, but it definitely produces undefined behavior. — John Bollinger, Sep 06 '15 at 03:20
The error checking in this code is not adequate. The result of every JNI API call needs to be checked, *immediately,* for a pending exception. Not several lines after you've used the result. Fix that and report any changed results or output here, in your question. — user207421, Sep 07 '15 at 00:07
@JohnBollinger thank you very much for looking at this. Your feedback led me to examine some things that eventually turned out to fix the problem (even though I'm still very confused). I greatly appreciate to your help and time! — codefinger, Sep 07 '15 at 02:29
@EJP Can you elaborate on this, or maybe link to an example that does have good error checking? Do i need to do more than check for null, or do I just need to make sure I'm checking for null after every call? Thanks for looking! — codefinger, Sep 07 '15 at 02:30
There are three exception-checking methods in the JNI API. You should call at least one of them after every JNI API call that can throw an exception. — user207421, Sep 07 '15 at 03:11
@EJP, I can't argue that one *shouldn't* call the exception-checking functions, but most JNI functions indicate failure *both* by raising an exception and by their return value (supposedly). For any of those functions, it is supposed to be sufficient to check the return value, which the original code in fact does pretty rigorously. I'd be very interested to hear about any of the JNI functions that are being used in this case raising an exception but also returning a value indicative of successful completion. — John Bollinger, Sep 07 '15 at 03:58
@JohnBollinger The original code calls *five* JNI APIs in a row, each one depending on the previous one's return value, and *then* starts checking the return values. And the first check just consists of comparing a function address to zero. You can't decribe that as 'pretty thorough'. Checking the return values and then calling the appropriate exception pending/describe method is not logically distinct from calling the appropriate exception pending/describe method, and not calling it all is simply wrong. — user207421, Sep 07 '15 at 09:33
@EJP, the code introduced by the edit matches your description, but the *original* code I was referring to is that at [the referenced GitHub page](https://github.com/jkutner/mruby-jvm/blob/master/src/java_support.c#L227), which is different. Looking over it again though, it appears he does miss exception checks for the calls at lines 288 and 291, which are analogous, but not identical, to the last two lines of the code from the question edit. — John Bollinger, Sep 07 '15 at 13:58

score -1 · Answer 1 · answered Sep 07 '15 at 02:27

-1

I have no idea why, but declaring this variable before the LoadLibrary call fixes the problem.

char stupid_var_that_means_nothing_but_makes_windows_work_i_dont_even[MAX_PATH];
HMODULE jvmdll = LoadLibrary(java_dl);

Commenting that line out causes the problem to start happening again. I've also tried adjusting it (changing the value in []) to no avail. I am completely stumped. I stumbled on this by accident after trying to add some code from jruby-launcher

Here is the full implementation of my JNI code.

I hate computers.

answered Sep 07 '15 at 02:27

codefinger

10,088
7
39
51

Keep looking. You've band-aided the problem, not solved it. – user207421 Sep 07 '15 at 03:11
@EJP yes, i will definitely continue my investigation. I'm hoping the error checking you mention above will lead to something. This answer deserves down-voting :) – codefinger Sep 08 '15 at 15:18

Why does this JNI call segfault on Windows only?

1 Answers1

Linked