Hair Analysis and Detection in Facial Picture

2011 December 26
by Emmanuel Goossaert

Continuing on my experimentations with face transformation algorithms, I was wondering if it was possible to detect and extract, with a certain accuracy, the hair of a person in a facial picture. I found two useful publications on the topic, Detection and Analysis of Hair (Yacoob and Davis, 2006), and Frequential and color analysis for hair mask segmentation (Rousset and Coulon, 2008). Their approaches are quite similar, and make use of color models, frequency filters and Gaussian kernels.

Based on these publications, I improved a bit the methods that they proposed by adding the use of the GrabCut algorithm. GrabCut does something similar to what the magic wand does in Photoshop: you feed it with some pixels, and it finds similar contiguous pixels. I am basically using the publications cited above to find some areas of the image that I am sure are hair. Then I feed these areas to GrabCut, and by iterating I can detect the hair with with more precision.

I implemented the whole thing in C++ with OpenCV. For GrabCut, I am using the implementation given in OpenCV, which is giving me good results. I have to admit however that I wouldn’t mind if it could be a bit faster. I am planning to release the code under an open source license at some point, but I would need to clean it up a little before I do so. [EDIT: 2014-06-25: The code was licensed in a commercial application and thus will not be released] Below are some examples of results, using pictures from the IMM Face Database (click on the pictures to enlarge). The last example is particularly interesting, as it shows that the detection is also successful in case of absence of hair πŸ™‚

30 Responses leave one →
  1. February 15, 2012

    Very good work. Can I have a copy of your code right now? Currently I am working on human detection that has some part relating to hair detection. Thanks in advance!

    • February 16, 2012

      Thank you for reaching out. I am sorry but I am very busy at the moment, and I won’t have the time to get the code clean and ready any time soon. I would advise that you take a look at the two publications I mentioned in the article, they are quite easy to implement with a basic knowledge of C++ and OpenCV. Also, I would be happy to answer your questions regarding the papers if you have any. From what I recall, it took me about three or four days to read the papers and implement the whole thing. But the real trick was the use of the GrabCut algorithm. Also, note I did not code GrabCut from scratch, I used the OpenCV implementation to save time.

  2. monkey permalink
    May 28, 2012

    Very good work. Can I have a copy of your code right now? Currently I am working on human detection that has some part relating to hair detection. Thanks in advance!

  3. GB Koo permalink
    July 19, 2012

    Dear Goossaert,
    It’s quite an excellent result.
    Actually I was looking for an expert in hair detection.
    If possible, I am highly interested in your help to develop our ” simplified hair syle detection model”. ( payable project)
    I would like to tell you more about it over the email.
    Please let me know your email or email me.

    GB

    • Wenshuang Tan permalink
      November 27, 2013

      Dear Goossaert,
      It’s quite an excellent result. May I ask you if the code is avaliable ? If yes, could you send me one copy?
      Thank you so much!

  4. September 24, 2012

    Hi,
    It has been a while, are you still considering opening up your code? We have a project coming up that could use the ability. Even if the code isn’t “ready” perhaps we could help turn it into something that is. Let us know, we would be glad to help. Send me an email if you would like us to “clean it” before you release it?
    Thanks!

  5. October 4, 2012

    Hi, the code is still not available.

    • Thiago permalink
      April 9, 2013

      Could you send me this code not cleaned? IΒ΄m study in University of the Espirito Santo (Brazil) and I need to do the same thing that their algorithm did. It is only to help a student, nothing commercial. I’ve got to capture webcam, already managed to make the detection the face and eyes, but I find difficulties using grabcut.
      If you can help I really do appreciate it.

  6. Walter permalink
    April 2, 2013

    Hi! Is it in your plans to make the code available? It’s been quite long time since you posted this examples and I was wondering if you could send me even an unpolished version of your running code.
    Thanks in Advance

    • Michiel Brinkers permalink
      October 16, 2013

      I would really appreciate that as well. Un unpolished version would be more than welcome πŸ™‚

  7. Alexandre da Silva permalink
    May 3, 2013

    Hi !!!
    I’m from Brazil, and I’m searching hair segmentation.
    I’m doing science of computating and I have a final course work, ok?
    Sorry, but I’m not perfect in english, I’m trying : P

    So, do you have a litle code of the GrabCut?
    If possible, send me email.

    Thnks !!!

  8. June 4, 2013

    @Alexandre:

    Here is a good link for the beginners on GrabCut.

    http://stackoverflow.com/questions/14111716/how-to-set-a-mask-image-for-grabcut-in-opencv

  9. Saleh permalink
    April 6, 2014

    Hi,
    good job πŸ˜€
    just wanna know that did you release the code?
    Thanks

  10. Gagan goel permalink
    May 23, 2014

    hi,
    nice work
    just wanted to know is the code available yet?

  11. May 28, 2014

    Can I get your algo for hair detection.
    I need it for my project.

  12. Akhila permalink
    May 30, 2014

    Hi Goossaert! Your work is really impressive. I am trying to implement it myself and just had some questions. I am implementing it in Python using OpenCV.

    My understanding (and correct me if I am wrong please!) is that the steps for Yacoob and Davis are:
    1) Face detection (easy using openCV)
    2) Eye detection (easy using openCV)
    3) Finding rectangles below each eye and on the forehead (straight forward, though I may need to play around with the size)
    4) Create a color model
    (This part is confusing me. How did you incorporate this with GrabCut? My understanding is GrabCut needs a mask that marks definite background pixels as 0 and definite foreground pixels as 1. )
    5) Make rectangles around the left, right and top (common areas where hair exists) and see if the pixels based on the color model are hair (again, straight forward, though I may need to play around with the size)
    6) Slide the rectangles up (This is where I assumed you added in GrabCut as you would now have a set of background pixels and a set of foreground pixels, yes?)

    My understanding of the steps for C. Rousset and P.Y. Coulon are:
    1) Face detection (again, easy using OpenCV)
    2) Defining head area (easy as the paper has extremely straight forward formulae)
    3) Frequential analysis
    (I don’t understand how to get the central frequency or the bandwidth. The paper says that the filter gives a frequential map. However, the frequential map takes in i and j (I assume these are just the coordinates of the pixel in the image) and the filter takes in a central frequency and a bandwidth. How do these two relate? Also what is f_theta? I understand the thresholding.)
    4) Morphomath treatment and labeling of artifacts (they did not go into detail for this. How did you go about doing this?)
    5) Color analysis – They defined a sample window at the top of the face. I assume this is simply a rectangle that is a bit above the face. Somehow they use the color information present in the frequential map, though I don’t know how or what exactly. They converted it to YCbCr color space. What confuses me next is that they use YCbCr color space in the calculation of the average color and the standard deviation, but then switch back to RGB for the color_mask equation.
    6) Use the logical and operator and intersect the two masks = fusion mask (very straight forward)
    7) Make a mask that puts 1 inside the fusion mask and 0 outside the frequential mask and inside the frequential mask but outside the “head area” (also straight forward)
    8) Use their alpha matting process (Is this where you did GrabCut?)

    C. Rousset and P.Y. Coulon seem to have better results. How exactly did you merge the two papers? Since Yacoob and Davis seemed more straight forward I have started to implement that.

    • June 25, 2014

      I wish I could help you but I haven’t worked on any of this for a long time now. My research interests have switched and I don’t have much time to dive back into all of this.

      Best luck with your project!

    • Navid permalink
      January 1, 2015

      Hi, any success?? I’m gonna implement this soon, let me know even if you got less than half of the way …

  13. July 9, 2014

    go to hell with your code

  14. Allen permalink
    September 10, 2015

    Why don’t you all who are all asking for source code get together and implement?

  15. Brian permalink
    November 28, 2015

    Hi, nice results. I wonder if you’ve tried it on images of people with dark skin and dark hair, and whether it still seems to work?

    Any comments/advice would be appreciated.

    Cheers

    • December 6, 2015

      Hey Brian! Yes, I did try the algorithm on a wide variation of hair and skin colors, and what came up for me is that the lighting conditions of the photos have a greater influence on the results than just hair or skin color. So to answer your question, yes the approach does work for dark skin and dark hair, and the limitations are the same as for light skin and light hair. Computer Vision “in the wild” is always a different story from the uniform backgrounds and uniform lighting conditions you find in the training sets of scientific publications.

      There is one paper that seems promising, Automatic Hair Detection in the Wild, 2010, by Julian, Dehais, Lauze, Charvillat [0]. Taking shapes into account as they do in this paper is definitely a good idea to separate the hair from the background, especially in cases where the background color happens to be very similar to the hair color in some locations. Note that I haven’t tried it so I can’t vouch for it, but that’s something worth reading.

      I hope this helps!

      [0] http://www.researchgate.net/profile/Francois_Lauze/publication/220929232_Automatic_Hair_Detection_in_the_Wild/links/0912f510760844d0c7000000.pdf

      • Brian permalink
        December 6, 2015

        Emmanuel, thank you so much for the pointers. I really appreciate you taking the time!

      • Tikam Chandrakar permalink
        December 21, 2015

        Please send me good link to understand the hair detection ,
        I need step by step process to implement on objective c
        iOS app.

  16. Marcos permalink
    January 21, 2016

    Hello, Emmanuel, nice work!!

    Well, I am interested in that commercial application…
    Is it still for sale, I could not found the link.
    Can you send me by email?

    Thanks.

    • January 21, 2016

      I licensed the library to a creative agency who needed it for a website they were making, and I signed a contract that prevents me from citing their name or linking to the application, so I can’t do that. The library I have works well for photos taken in a controlled environment, that is to say when photos have a clean background and normalized luminosity, but works less well with photos taken “in the wild”, when the background is random and the luminosity messed up. The creative agency did not need perfect hair segmentation, they only wanted to detect the hair size of people in photos in the wild, i.e. bold, very short, short, medium, and long hair, which is a simpler problem, and the library did an acceptable job at that.

      • Vladimir permalink
        May 29, 2016

        Can u sell your library for another commercial project? We need to cut hairs plus face from image, and we can ask users to use plain background. Can we discuss price and opportunity?

  17. Rajibul Alam permalink
    May 10, 2016

    πŸ™‚ Manu impressive work, glad you kept on working with AI (image processing in this case)

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS