hello everyone this is Fahad mza and in this video we are going to cover the tool marker we will see how to install it locally and then how can we convert our complex PDF files to markdown Json or HTML format we already have covered this marker tool when it was released around 7 to 8 months ago as you can see on the channel but for the purpose of this video we are going to install it again because there have been lot of improvements around it plus they also have a new graphical user interface backed by streamlit which we will be installing and trying it out on our complex PDF file which I will also show you if you look at the GitHub repo of this marker it has come a long way certainly but the good thing is that they are not trying to do everything in one place they are keeping to their promise of doing only PDF to markdown or Json HTML which is quite good they're not trying to deal with Word documents PowerPoint and lot of other things which I think makes sure that the quality remains there we already have covered similar tools like talkling market toown nart and few others on the channel if you're interested you can search on my channel but this marker tool only deals with PDF as I said and Deals it quite nicely and I already have C cover the comparison of this tool with others now not only it deals with PDF but it is sort of a pipeline of deep learning models that extract text it also does optical character recognition from PDF if necessary plus it detects the page layout and find a reading order the good thing which I really like about marker is that it produces a clean and format formatted block from the PDF and then it combines those blocks and postprocesses a complete text and it does uses some of its own models behind the scene but only when necessary so it is quite easy to use it on your CPU machines GPU is not mandatory here if you're looking to use it commercially there are some licensing constraint but most of the time it is quite liberal for example if your organization has uh $5 million USD in Gross revenue or under you can use it for I think uh for some period of time and then um there is a lot of other liberal concessions which you can check on the GitHub repo and I will drop the link to it in videos description now we all know that PDF is a tricky format so Mar marker will not always work perfectly for example it will only convert block equations tables are not always format 100% correctly if you're dealing with a lot of tabular data in PDFs this is not the tool use dock link for it much much better and then especially uh and also if you're using forms on PDF again doesn't work dock Ling Works quite nicely so very so if you so here is a decision tree if your PDF contains lot of tables lot of forms and complex one mark is not the tool for you other than that simple PDFs with some of the basic tables and forms not not a bad tool at all so let's try to get it installed and then we will see how it works let me also give a huge thanks to M compute who are sponsoring the VM and GPU for this video If you're looking to rent a GPU on very very affordable prices I will drop the link to their website in video's description plus I'm also going to give you a coupon code of 50% discount on range of GPU so do check them out this is my one 12 system and for full disclosure I'm using the GPU sponsored by mast compute which is NVIDIA rtxa 6000 with 48 GB of vram let's try to install it I'm first creating this virtual environment with K and next up I'm installing the Cuda so that if there is a GPU usage it is is going to use my GPU and then most importantly I'm installing torch and torch Vision if you don't have a GPU you can skip the first two commands and only install torch and torch Vision this is going to take a minute or so and now let's install marker PDF with streamlit and it's a simple pip command that installs all the auxiliary things including pente and few other stuff like tqdm on our local system system so let's wait for it to get installed and you see that it is quite lightweight shouldn't take too long and the installation is done let me clear the screen and you can then launch the GUI of marker by just running this marker _ GUI and it is going to open this stream late demo on your browser first time when you run it it takes bit of a time to initialize so let's wait and you can also check it out on here you see that first time it also downloads these models you can also check it out here that load model module is running it's not you can see that the model is not that big and there you go took around 5 minutes to download everything and our Market demo is running also let me quickly show you the vram consumption to see if how it is consuming on my GPU so just close to 3 gig of vram it is using what as I said you can run it on CPU easily okay so it's very very simple straightforward on the left hand side you can browse your file and the file size should not be more than 200 MP you can of course select multiple files now the file which I'm going to use for it is this it's a doc link technical report and the reason why I have selected it is that it contains some of the well formatted text some scientific notations images and also some tabular data and few other stuff like annotations and references so it's a good mix of different sort of things which we can expect in a PDF file nine pages so let me select it it from my local system I think it is in my my files PDF and this is the one and you you can see that it has loaded the file on the right hand side and then it is saying that this will um going to converted to the markdown now you can go with the whole stuff 0 to zero or you could just even give it some of the pages like 0 to 0 to 5 0 to 10 and all that stuff you can select the format to CH on our HTML I'll just go with markdown you can force so here and and then you know it will just do OC on everything I'm not doing it in this one and then maybe I'll just say okay let's go from 0o to five how is that and then I will just click on and press enter to apply okay let's go here and let's click on run marker it is running as you can see on the top it already is doing this there you go you see how cool is that and how quick is that this is your markdown format 0 to 5 Pages let's go there so it has done the basic table to images to for the and then all the references how good is that how good is that really really good stuff and then next up I have selected the Json format let's click on run marker and then you can see on the right hand side it has created it the Json format with all the data and everything it's quite huge so this is more useful for if you're trying to run it for the uh programmatic data set generation or something like that similarly you can select it for for the HTML and then you can click on marker again it is running let's wait for it there you go so this is our HTML you can open it in the browser to see there you how good this looks and for this one it hasn't given us the images that is only for the uh first one which is a markdown really good stuffff and then if you're facing any issues you can uh just select this tbug it is going to show you the tbug so now let me run HTML with for to see if it is able to do the images or not or if it is only for the markdown format let's wait for it I'm letting it run so that you would also see the speed in real time there you go so let me scroll down there you go so images are only on the markdown but other than that looks really cool I don't see any issues there table table format is also there good stuff so you see looks really cool to me and I think it has improved a lot also not only in GUI you can also use the CLI with it you can even use it in your python code quite easily and I already have covered that in my other video which is a second video as you can see here as how you can use it with CLI so hopefully you enjoyed it let me know what do you think about this I will drop the link link to it in video description if you like the content please consider subscribing to the channel if you are already subscribed please share it among your network as it helps thank you for watching