When you give a variable to case, it's compared against each when clause with the === method. The first === to return true will be the branch that's executed.
In your case, number < 0 is evaluating to true, and -5 === true is false!
What you want to do is either leave the number off of the case, and make all of the when clauses boolean:
case
when number < 0
return "Please enter a number greater than 0."
when number.between?(0, 1)
return false
when number == 2
return true
end
Or leave the number on, but make all of the whens values that you can compare against:
case number
when -Float::INFINITY..0 # or (-1.0/0)..0
return "Please enter a number greater than 0."
when 0..1
return false
when 2
return true
end