You'll want to make a common denominator with 6 and 8. That denominator would be 24. 24/6=4 so you would have to multiply 5/6 by 4/4 to get 20/24. Next, 24/8=3 so 1/8 could be multiplied by 3/3 to get 3/24. Since 3 is less than 20, 1/8 is smaller than 5/6.
If you want the same numerator, 5/8 = 15/24. This would make 5/8 smaller than 5/6 as well. The highest eighth you can go is 6/8 which is 18/24. So you can use any numerator between 1 and 6 with a denominator of 8 to get a fraction smaller than 5/6.