Continuing on my experimentations with face transformation algorithms, I was wondering if it was possible to detect and extract, with a certain accuracy, the hair of a person in a facial picture. I found two useful publications on the topic, Detection and Analysis of Hair (Yacoob and Davis, 2006), and Frequential and color analysis for hair mask segmentation (Rousset and Coulon, 2008). Their approaches are quite similar, and make use of color models, frequency filters and Gaussian kernels.
Based on these publications, I improved a bit the methods that they proposed by adding the use of the GrabCut algorithm. GrabCut does something similar to what the magic wand does in Photoshop: you feed it with some pixels, and it finds similar contiguous pixels. I am basically using the publications cited above to find some areas of the image that I am sure are hair. Then I feed these areas to GrabCut, and by iterating I can detect the hair with with more precision.
I implemented the whole thing in C++ with OpenCV. For GrabCut, I am using the implementation given in OpenCV, which is giving me good results. I have to admit however that I wouldn’t mind if it could be a bit faster. I am planning to release the code under an open source license at some point, but I would need to clean it up a little before I do so. [EDIT: 2014-06-25: The code was licensed in a commercial application and thus will not be released] Below are some examples of results, using pictures from the IMM Face Database (click on the pictures to enlarge). The last example is particularly interesting, as it shows that the detection is also successful in case of absence of hair π
Very good work. Can I have a copy of your code right now? Currently I am working on human detection that has some part relating to hair detection. Thanks in advance!
Thank you for reaching out. I am sorry but I am very busy at the moment, and I won’t have the time to get the code clean and ready any time soon. I would advise that you take a look at the two publications I mentioned in the article, they are quite easy to implement with a basic knowledge of C++ and OpenCV. Also, I would be happy to answer your questions regarding the papers if you have any. From what I recall, it took me about three or four days to read the papers and implement the whole thing. But the real trick was the use of the GrabCut algorithm. Also, note I did not code GrabCut from scratch, I used the OpenCV implementation to save time.
Very good work. Can I have a copy of your code right now? Currently I am working on human detection that has some part relating to hair detection. Thanks in advance!
Hello. The code is still not available.
Dear Goossaert,
It’s quite an excellent result.
Actually I was looking for an expert in hair detection.
If possible, I am highly interested in your help to develop our ” simplified hair syle detection model”. ( payable project)
I would like to tell you more about it over the email.
Please let me know your email or email me.
GB
Dear Goossaert,
Itβs quite an excellent result. May I ask you if the code is avaliable ? If yes, could you send me one copy?
Thank you so much!
Hello Goossaert
I am looking for hair detection to make them color. can i have your help to do it as soon as possible i am stuck in that so plaease help me out
Regards
Hi,
It has been a while, are you still considering opening up your code? We have a project coming up that could use the ability. Even if the code isn’t “ready” perhaps we could help turn it into something that is. Let us know, we would be glad to help. Send me an email if you would like us to “clean it” before you release it?
Thanks!
Hi, the code is still not available.
Could you send me this code not cleaned? IΒ΄m study in University of the Espirito Santo (Brazil) and I need to do the same thing that their algorithm did. It is only to help a student, nothing commercial. I’ve got to capture webcam, already managed to make the detection the face and eyes, but I find difficulties using grabcut.
If you can help I really do appreciate it.
Hi! Is it in your plans to make the code available? It’s been quite long time since you posted this examples and I was wondering if you could send me even an unpolished version of your running code.
Thanks in Advance
I would really appreciate that as well. Un unpolished version would be more than welcome π
Hi !!!
I’m from Brazil, and I’m searching hair segmentation.
I’m doing science of computating and I have a final course work, ok?
Sorry, but I’m not perfect in english, I’m trying : P
So, do you have a litle code of the GrabCut?
If possible, send me email.
Thnks !!!
@Alexandre:
Here is a good link for the beginners on GrabCut.
http://stackoverflow.com/questions/14111716/how-to-set-a-mask-image-for-grabcut-in-opencv
Hi,
good job π
just wanna know that did you release the code?
Thanks
hi,
nice work
just wanted to know is the code available yet?
Can I get your algo for hair detection.
I need it for my project.
Hi Goossaert! Your work is really impressive. I am trying to implement it myself and just had some questions. I am implementing it in Python using OpenCV.
My understanding (and correct me if I am wrong please!) is that the steps for Yacoob and Davis are:
1) Face detection (easy using openCV)
2) Eye detection (easy using openCV)
3) Finding rectangles below each eye and on the forehead (straight forward, though I may need to play around with the size)
4) Create a color model
(This part is confusing me. How did you incorporate this with GrabCut? My understanding is GrabCut needs a mask that marks definite background pixels as 0 and definite foreground pixels as 1. )
5) Make rectangles around the left, right and top (common areas where hair exists) and see if the pixels based on the color model are hair (again, straight forward, though I may need to play around with the size)
6) Slide the rectangles up (This is where I assumed you added in GrabCut as you would now have a set of background pixels and a set of foreground pixels, yes?)
My understanding of the steps for C. Rousset and P.Y. Coulon are:
1) Face detection (again, easy using OpenCV)
2) Defining head area (easy as the paper has extremely straight forward formulae)
3) Frequential analysis
(I don’t understand how to get the central frequency or the bandwidth. The paper says that the filter gives a frequential map. However, the frequential map takes in i and j (I assume these are just the coordinates of the pixel in the image) and the filter takes in a central frequency and a bandwidth. How do these two relate? Also what is f_theta? I understand the thresholding.)
4) Morphomath treatment and labeling of artifacts (they did not go into detail for this. How did you go about doing this?)
5) Color analysis – They defined a sample window at the top of the face. I assume this is simply a rectangle that is a bit above the face. Somehow they use the color information present in the frequential map, though I don’t know how or what exactly. They converted it to YCbCr color space. What confuses me next is that they use YCbCr color space in the calculation of the average color and the standard deviation, but then switch back to RGB for the color_mask equation.
6) Use the logical and operator and intersect the two masks = fusion mask (very straight forward)
7) Make a mask that puts 1 inside the fusion mask and 0 outside the frequential mask and inside the frequential mask but outside the “head area” (also straight forward)
8) Use their alpha matting process (Is this where you did GrabCut?)
C. Rousset and P.Y. Coulon seem to have better results. How exactly did you merge the two papers? Since Yacoob and Davis seemed more straight forward I have started to implement that.
I wish I could help you but I haven’t worked on any of this for a long time now. My research interests have switched and I don’t have much time to dive back into all of this.
Best luck with your project!
Hi, any success?? I’m gonna implement this soon, let me know even if you got less than half of the way …
go to hell with your code
Why don’t you all who are all asking for source code get together and implement?
Hi, nice results. I wonder if you’ve tried it on images of people with dark skin and dark hair, and whether it still seems to work?
Any comments/advice would be appreciated.
Cheers
Hey Brian! Yes, I did try the algorithm on a wide variation of hair and skin colors, and what came up for me is that the lighting conditions of the photos have a greater influence on the results than just hair or skin color. So to answer your question, yes the approach does work for dark skin and dark hair, and the limitations are the same as for light skin and light hair. Computer Vision “in the wild” is always a different story from the uniform backgrounds and uniform lighting conditions you find in the training sets of scientific publications.
There is one paper that seems promising, Automatic Hair Detection in the Wild, 2010, by Julian, Dehais, Lauze, Charvillat [0]. Taking shapes into account as they do in this paper is definitely a good idea to separate the hair from the background, especially in cases where the background color happens to be very similar to the hair color in some locations. Note that I haven’t tried it so I can’t vouch for it, but that’s something worth reading.
I hope this helps!
[0] http://www.researchgate.net/profile/Francois_Lauze/publication/220929232_Automatic_Hair_Detection_in_the_Wild/links/0912f510760844d0c7000000.pdf
Emmanuel, thank you so much for the pointers. I really appreciate you taking the time!
Please send me good link to understand the hair detection ,
I need step by step process to implement on objective c
iOS app.
Hello, Emmanuel, nice work!!
Well, I am interested in that commercial application…
Is it still for sale, I could not found the link.
Can you send me by email?
Thanks.
I licensed the library to a creative agency who needed it for a website they were making, and I signed a contract that prevents me from citing their name or linking to the application, so I can’t do that. The library I have works well for photos taken in a controlled environment, that is to say when photos have a clean background and normalized luminosity, but works less well with photos taken “in the wild”, when the background is random and the luminosity messed up. The creative agency did not need perfect hair segmentation, they only wanted to detect the hair size of people in photos in the wild, i.e. bold, very short, short, medium, and long hair, which is a simpler problem, and the library did an acceptable job at that.
Can u sell your library for another commercial project? We need to cut hairs plus face from image, and we can ask users to use plain background. Can we discuss price and opportunity?
π Manu impressive work, glad you kept on working with AI (image processing in this case)
Hi,
Can you please tell me what you used from the paper to get the hair.
For Grabcut did you define each hair pixel or was the matting process done by grabcut itself?
hey nice work can i get source code for it??
i want one android project witch can result me hair detection
thank you
Hi Emmanuel Goossaert,
nice work, I wonder if you can still see this after such a long time, but I just have a small question:
when you try to detect and extract the hair, “feed it with some pixels” means you still need to draw “strokes” on the original pixel, not automatic, right?
Thank you
This is a fake, guys