LogFAQs > #968447112

So what you are probably noticing on 50 is that it is the first episode to not be based on manga pages. It's not true filler though as Oda says the stuff with Daddy was something he wanted to include but it ended up cut from the manga.

51 is basically just filler. 52 is back to canon.

54-60 is the first true filler arc of the series if you want to skip it. Though 55 has a bit of canon and 61 has a tiny bit of the filler arc. It's not bad just mediocre imo.