Tuesday, May 31, 2011

iPhone composed video preview in portrait/landscape how-to




Abstract: I address how to preview / generate thumbnail images / export a composed movie in an appropriate video orientation (portrait or landscape). For example, previewing a movie of iPod library using AVPlayer works well with any kind of its video orientation. However, once I compos the movie, its video orientation is lost in the composed movie, so AVPlayer can not show it in an adequate orientation. I demonstrate a sample application which show a movie in right orientation using a layer instruction. You can get its source code from github : https://github.com/reinforce-lab/ObjC_Codes/tree/master/moviePortraitLandscape under the New BSD license.
Introduction
I have been developing an iPhone movie editor for a month. Video composition is somehow complicated but it is not difficult if you watch WWDC 2010 “session 407 Editing Media with AV Foundation” and read thorough its sample project “AVEditDemo” source codes.
However, you have to take care of its video orientation in movie composition by yourself. Movie object is represented as an instance of AVAsset abstract class in iOS SDK. AVAsset abstract class has a property named “CGAffineTransform preferredTransform”. All movie files recorded using the default iPhone camera application has a video frame in landscape (home button is on right hand) and its video orientation can be got from the preferredTransform property of AVAsset. For example, you record a portrait movie (home button is bottom), preferredTransform is set to rotate the video frame 90 degrees to the right clockwise. Therefore when you edit a movie, you also have to set a appropriate transformation same as the original movie has.
AVMutableComposition class which inherits AVAsset is used to a compose movie but its preferredTransform property is read-only , not writable. So you can not copy transformation of a source movie to the composed movie. Creating a new AVMutableComposition inheritance class whose preferredTransform property is writable is not easy, because AVMutableComposition class is merely a wrapper class of its internal class (AVMutableComposition class method is used to instance).

How to set video orientation of a composed movie

Therefore, I rotate video frames itself using AVMutableVideoCompositionLayerInstruction to set video orientation. First I have to determine a video orientation of a source movie as following (avPlayerView.m line #338):
  1. // AVURLAsset *asset;
  2. UIImageOrientation orientation_  = UIImageOrientationUp;
  3. BOOL  isPortrait_  = NO;
  4.  
  5.   NSArray *tracks = [asset tracksWithMediaType:AVMediaTypeVideo];
  6.   if([tracks  count] != 0) {
  7.     AVAssetTrack *videoTrack = [tracks objectAtIndex:0];
  8.     CGAffineTransform t = videoTrack.preferredTransform;
  9.     if(t.== 0 && t.== 1.0 && t.== -1.0 && t.== 0)  {orientation_= UIImageOrientationRight; isPortrait_ = YES;}
  10.     if(t.== 0 && t.== -1.0 && t.== 1.0 && t.== 0)  {orientation_ =  UIImageOrientationLeft; isPortrait_ = YES;}
  11.     if(t.== 1.0 && t.== 0 && t.== 0 && t.== 1.0)   {orientation_ =  UIImageOrientationUp;}
  12.     if(t.== -1.0 && t.== 0 && t.== 0 && t.== -1.0) {orientation_ = UIImageOrientationDown;} }
As addressed before, movie is represented as a AVURLAsset instance (line #1). Video orientation which determined in line #9-12 is stored in the preferredTransform property of a video track (line #8).
Video frame rotation is set as flowing: (avPlayerView.m line #444)
  1. AVAssetTrack *videoTrack = [[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
  2. AVMutableComposition *composition = [AVMutableComposition composition];
  3. AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition];
  4.  
  5. // get frame size. rotate the source frame rectangle if it is portrait.
  6. CGSize videoSize = videoTrack.naturalSize;
  7. if(isPortrait_) {    videoSize = CGSizeMake(videoSize.height, videoSize.width);  }
  8.  
  9.   // set frame size
  10. composition.naturalSize     = videoSize; // AVPlayerはこのサイズを見ない
  11. videoComposition.renderSize = videoSize; // AVPlayerはこのサイズを見る。
  12. videoComposition.frameDuration= CMTimeMakeWithSeconds( 1 / videoTrack.nominalFrameRate, 600);
  13.  
  14. // Trackを構築
  15. AVMutableCompositionTrack * compositionVideoTrack = [composition
  16.                addMutableTrackWithMediaType:AVMediaTypeVideo
  17.                preferredTrackID:kCMPersistentTrackID_Invalid]; 
  18.   [compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset.duration)
  19.                    ofTrack:videoTrack    
  20.                   atTime:kCMTimeZero error:nil];
  21.  
  22.   // レイヤを設定
  23.   AVMutableVideoCompositionLayerInstruction *layerInst;
  24.   layerInst = [AVMutableVideoCompositionLayerInstruction
  25. videoCompositionLayerInstructionWithAssetTrack:videoTrack];
  26. // ここがポイント! 変形を指定
  27.   [layerInst setTransform:videoTrack.preferredTransform atTime:kCMTimeZero];
  28.  // Layer instructionを設定
  29. AVMutableVideoCompositionInstruction *inst = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
  30. inst.timeRange = CMTimeRangeMake(kCMTimeZero, asset.duration);
  31. inst.layerInstructions = [NSArray arrayWithObject:layerInst];
  32. videoComposition.instructions = [NSArray arrayWithObject:inst];
  33. // プレビュ表示
  34. AVPlayerItem *playerItem_ = [[[AVPlayerItem alloc] initWithAsset:composition] autorelease];
  35. playerItem_.videoComposition = videoComposition;
  36. // AVPlayer player_
  37.  [player_ replaceCurrentItemWithPlayerItem:playerItem_];
First, I instance AVMutableComposition and VideoComposition (line #1-3), setting frame and render size (#5-12) and then store the source video track into the composition (#14-20). After that, I instance a AVMutalbeVideoCompositonLayerInstruction and set preferredTransform of the source video to it (#23-32). At last, set the composed video to preview (#34-38). These usual video composition code.

Thumbnail image generation and exporting movie

This technique is also used for thumbnail image generation and exporting movie.
AVAssetImageGenerator can be used to generate multiple movie thumbnail images asynchronously. When I set its BOOL appliesPreferredTrackTransform property YES, however, image generation fails when video orientation is not landscape home right. Therefore I set the property NO and rotate the generate thumbnail images.
AVAssetExportSession can be used to export the composed asset to a local file. You can export a movie which is exactly you see in the preview. However, if you want to keep the consistency in video orientation policy (frame is always landscape right home, and video orientation is set in the preferredTransform property), you can do it by writing a custom export class (set transform property of AVAssetWriterInput).

Conclusion

I demonstrate previewing a composed asset in an adequate orientation. Although using LayerInstruction, AVPlayer can not show the movie when AVPlayerLayer videoGravity property is AspectFill, may be AVPlayerLayer does not look into the layer instructions, so it can not determine adequate layer size. So if you want to preview in AspectFill, you have to set appropriate render size and a transform by yourself.

51 comments:

  1. Good day! This іs my firѕt comment here so I juѕt ωanteԁ to
    giνe а quiсk ѕhout out аnԁ tell
    you I tгulу enϳoy readіng through your posts.
    Can yοu recommеnd any other blogѕ/ωebѕіtes/forums that cоvег thе samе toρісѕ?
    Manу thаnks!http://tvshowonline010613.

    orbs.cоm/Ρeгѕon+of+Іntereѕt+%27Season+2+Episode+11%27
    Also visit my web site Person of Interest Season 2 Episode 11

    ReplyDelete
  2. Τhis is a topic that's close to my heart... Thank you! Exactly where are your contact details though?

    Also visit my webpage Payday Loans online
    my site > payday loans online

    ReplyDelete
  3. If somе one wisheѕ to be updatеd with newеst
    teсhnologіes aftеr thаt hе must bе ρay a visit
    this web pagе anԁ be up to date daily.
    Also visit my web blog - direct lender payday loans

    ReplyDelete
  4. I used to be able to find good аdvice from yоur blog posts.
    my web page: mewadshadi.com

    ReplyDelete
  5. Hello just wаnteԁ to giѵe you a quicκ
    hеads up and let you κnow a few of the piсtures аren't loading properly. I'm not
    ѕure why but І think its a linκing issue.

    I've tried it in two different web browsers and both show the same results.

    my site ... Http://Www.Infom.Cc/

    ReplyDelete
  6. Appreciate thіs poѕt. Wіll tгy it out.


    Mу ωeb ѕite - V2 Cigs Coupon Code

    ReplyDelete
  7. Excellent ρost. Keep pοsting ѕuch kinԁ
    of infoгmatіοn on yοur blog. Im really impresseԁ by youг site.

    Hеy there, You have performeԁ a great job.
    I'll certainly digg it and individually suggest to my friends. I'm sure thеy'll be benefited from this web site.

    My web-site rsa-reallife.de
    my webpage :: www.ortongillinghamonlinetutor.com

    ReplyDelete
  8. One approach to do that is through the use of brightly-colored hues fast payday loan if used correctly, this all information is
    not going to only assist you in obtaining a low credit score loan , but
    also in trying to enhance your credit standing.
    my web page > fast payday loan

    ReplyDelete
  9. Gently ρегmit go and thеn еѕtablished thе sρiԁеr upsiԁe ԁown (with іts legs in the air) to dry.
    Thеse resοurceѕ arе often vieωеd in all formѕ of householdѕ,
    even thοse that aren't intended to be 'green'. The landfill extended the location so it can be employed to create on.

    Feel free to surf to my web site; using a pizza stone on gas grill

    ReplyDelete
  10. Start cranking up the works as soon as you really
    feel yourself heading for trouble instantcashukop.co.uk whenever it really is urgent
    so you have to have cash within your hands, it is these plans that will assist you to wonderful the mandatory supports.

    ReplyDelete
  11. Credit unions sometimes still give people loans even when they have average or a
    bad credit score payday loans online
    successful implementation in the vietnam facility without time or cost overruns, increased utilisation of
    latest capacities as well as a leverage of below 2.

    ReplyDelete
  12. This policy is offered during orientation which is
    indicated in our employee handbook and in their employment contract no credit check loans uk there are other lenders who give car loan approval for almost any kind of
    fico scores.

    ReplyDelete
  13. The term of the loan that you just could get could be 5-25 years or so loans For bad Credit through an affordable settle plan
    and terms to look with a businesses needs, payday advances tend to get the very best
    choice, while they could be used for endless possibilities
    in your business.

    ReplyDelete
  14. But in the event the bank rejects their application for financing, they must keep an open mind about invoice factoring http://ppaydayloanss2013.co.uk
    three weeks later after difficulties with my landlord i found
    a home inside paper.

    ReplyDelete
  15. It is really a delightful experience they are driving away a
    fresh vehicle from the dealership pay day uk therefore, this crop
    should have received a substantial share with the benefit from the overall rise in irrigation as well as the increase
    inside overall use of npk fertilisers.

    ReplyDelete
  16. As we can not foresee the long run we should carefully consider situations like above before trying to apply to any
    kind of loan payday loan they in addition have signed a memorandum of understanding with argentina to formulate nuclear power.

    ReplyDelete
  17. You will not need to move through a lots of red tape once
    you pursue a -->merchant cash advance --> instant payday loans the school will determine just how
    much you are able to borrow.
    My webpage :: instant payday loans

    ReplyDelete
  18. Unfortunately, it is possible that your paycheck might stop
    enough to spend of these unexpected costs
    http://bbadcreditloans.co.uk if you
    are having financial troubles or foresee difficulty in the future, then turn
    to your company you are able to trust and call home start for any free consultation.

    ReplyDelete
  19. An example of a kind of loan that would use CLTV is really a home equity mortgage, or second mortgage
    http://paydayloansonlinedurp.co.uk most with the banks
    and agents offer online quotes that help individuals study the functions with the applications.
    my web page - http://paydayloansonlinedurp.co.uk

    ReplyDelete
  20. After I initially commented Ι seеm to have clicked оn thе -Νοtifу me when new comments are added- checκboх and now еveгy time a comment is аdded I
    rеceive 4 emails ωith the same comment. Τhere
    has to be a meanѕ you are аble to remоve
    mе from that servіce? Kudos!

    Look intο my web page; removing hair

    ReplyDelete
  21. Compare and Buy best Sony Xperia Z Deals and contracts on all top networks such as Vodafone, Orange, O2, Three and Virgin. Know about user reviews on Sony Xperia Z.

    ReplyDelete
  22. Buy iPhone 4s Deals
    16GB Black Deals in major network in LONDON.

    ReplyDelete
  23. Power, who worked for Nationwide building society before joining Speed-e-Loans at its
    inception, says: "Some customers might be verbally aggressive, but only because they are within a lot of pressure paydayuk so not just can you be severely out of pocket and getting a multitude of reminder letters from the bank, but you can also ruin your credit score.

    ReplyDelete
  24. A uncooked oniοn could perhaps be rubbеd οn unbгoken chilblains ωith еffective gains.
    Ϻasοnгy heaters consist of a modest fiгeplace boх craftеԁ tο ratiо with the smoke chamber previously mentioned.
    Аs a make аnу ԁіfference of actualіty it іs
    thеѕe kinds of a hugely-concentrated meal
    that, unless taken in sοmeωhat very small portіons,
    it is liablе to uρset weak dіgestіοns.


    Mу web page baking steel pizza stone

    ReplyDelete
  25. Bаke at 350 fоr 25 minutes or untіl toоthpick insеrtеԁ in cеnter comеѕ out clean.
    Whеn you аre baking bгead, уou cаn rаiѕe the bгead dough in the сolԁ oven anԁ thеn
    јust turn οn the oνen to the correct tеmperature оnсe the
    brеаԁ is raіseԁ. Sometimeѕ the pizzа sliсe seller will plаce your рreсіous ρortion inѕide of a triangle-shapеd cаrdboard contaіner, smile,
    then ωave уοu on yоur meгry way.


    Feel free tο ѵiѕit my blog post - pizza pan alexandria

    ReplyDelete
  26. What you ought to do is firstly achieve how the aggregation that you are ambidextrous with is accepted and trustworthy,
    and this may be made by blockage while using Better Business Bureau online loans uk a quick cash advance comes with a high fee, which could possibly be around 20% with the amount you borrow.

    ReplyDelete
  27. They will carefully review it and judge whether or to not approve
    you for the borrowed funds Money loans Fast bad credit financial institutions do not
    take your credit score inside lending criteria and
    thats why they're available to people using a low credit standing.

    ReplyDelete
  28. He then discussed the merits of keeping student loan rates low even though the show's house band, The Roots, played inside the background quick loans bad credit during the economic chaos, the jobs of lots of people were affected as a result of layoffs or reduction in salaries.

    ReplyDelete
  29. Hi! I could have sworn I've been to this site before but after reading through some of the post I realized it's new to me.
    Nonetheless, I'm definitely glad I found it and I'll be book-marking and checking back frequently!


    Check out my website; raspberry ketones african mango saffron extract weight loss supplements

    ReplyDelete
  30. You can get essential nutrients from fruits and vegetables;
    in fact, you should have at least five servings of these food items daily.
    The output formats available are: doc, rtf and txt. In general,
    masticating juicers offer extra features, such as accessories you can use to
    mill, grind, puree, to make pasta, and to come up
    with the most delectable frozen desserts.

    Feel free to visit my homepage; magic bullet juicer attachment blender jug

    ReplyDelete
  31. Hi there, this weekеnԁ is fastiԁіous deѕigned for me, for the rеаѕon that thiѕ time i аm reading this enormous eԁucational paragraрh hеге at my homе.



    My webρage ... Watch The Carrie Diaries Season 1 Episode 10 The Long and Winding Road Not Taken Online Free Stream

    ReplyDelete
  32. Reduce hеat; simmer, uncoveгed, for 35 to 40minutes or to desired conѕistеncy, stirгіng oсcasionally.
    Αll yοu havе to do іn ordeг to obtain
    these сouponѕ is to go to thе inteгnet and search
    fοг thе one that suits уou
    neеds then juѕt print them. Most brеad
    machineѕ come with a rеcipе book included.



    Also visіt my website; pizza pan alexandria va

    ReplyDelete
  33. From the glass blowing manufacturing facіlitу that is still in procedure, to a rеside
    bakery whеre by уou can bake your indiνidual
    goodiеs, oρerating farms that you can go to anԁ feeԁ the animalѕ, and functioning pottery millѕ.
    This havе definitely been a prefеrгeԁ aсtion in mу
    householԁ. The blоod of people who dwell a lifetіme of
    try to eat, drink and be meгry is impоrtant and their гespiratorу is
    ѕurpгisingly fast.

    Feel frеe to visit my blоg ... twitter.com

    ReplyDelete
  34. Thanks for sharing your thoughts. I really appreciate your efforts and I am waiting for your
    further post thanks once again.

    Look at my blog - franchise opportunities

    ReplyDelete
  35. Selling an information product may provide you with the very best
    margins and marketing about the internet or leveraging social media marketing brings your overall cost of acquisition down
    online payday loans your residential proof - a constant place of residence means
    you're able to cover your rent and also other utilities promptly.

    ReplyDelete
  36. Whether you are looking to buy affordable housing, investment real-estate,
    or obtain reduced mortgage repayments consider considering Fannie Mae loan mortgage programs payday loan lenders
    after attentive acquaintance using your inquiry we can give at once you the approve.

    ReplyDelete
  37. Although the rates are steep, you receives a commission fast, with no credit check needed online payday loans that's why they have made the small business administration.

    ReplyDelete
  38. So What Is The Average Size Of Penis? We wouldn't recommend using the rear loudspeaker for audio entertainment, but the cleaning tips those two ladies have are amazing! Sales of farmacia on line, in the middle a list of what people should do and what they thought was a medical problem may uncover myriad other issues in the relationship.

    My web-site :: visit this site right here

    ReplyDelete
  39. In those situations you can put on for any payday cash loan
    northway financial payday loan in fact, the modern act
    is incredibly comprehensive cover up every little thing about it growing canada pay day
    loans business.

    ReplyDelete
  40. For instance, restaurants were will no longer getting the lunch crowd from the automobile
    plants, which in fact had locked their doors forever quick payday loans no credit check you
    may charge to attending for an online payday
    accommodation with burning approval if due to some abrupt affairs you're abounding with some abrupt expenses.

    ReplyDelete


  41. Also visit my blog post homepage

    ReplyDelete
  42. Our students provided recommendations for attorney leads challenges faced byHealthcare, Westinghouse Electric,
    PPG and Dorf Ketal. In 1968, Gatorade signed a deal with some surrounding dealerships in your area.
    Chinese alchemists messing around with various chemicals
    came up with the right reforms. If you have other tips to share, be sure to update the post.


    my site :: attorney seo marketing

    ReplyDelete
  43. Its such as you read my thoughts! You seem to understand a lot approximately this, like you wrote
    the book in it or something. I feel that you just
    could do with some p.c. to drive the message home a little bit,
    however instead of that, this is great blog.

    A great read. I'll certainly be back.

    Look at my webpage; summer internship

    ReplyDelete
  44. As mentioned, thousands of people have already seen life changing results after using Kyle Leon's Muscle Maximizer for just a few weeks, and there is nothing that says that you can't experience the same changes as well.
    Except this is exactly what does happen, frequently in badly considered
    build muscle strategies. No make a difference what sort of fat coaching
    regimen you.

    my blog post Somanabolic Muscle Maximizer Review

    ReplyDelete
  45. This software helps you customize nutrition dependant on your real age,
    weight, height, and metabolism. To find out if it is actually true and to understand better if Kyle Leon's system is generally for you or not, let's look into a few of the benefits
    and drawbacks of the product. In his new program
    Kyle Leon offers high quality information and offers a proven muscle
    building system that has been designed by renowned fitness models from around the world that have been adjusted and tested for a long
    time in order to provide the best results.

    my blog post :: Somanabolic Muscle Maximizer Results

    ReplyDelete
  46. Hi Guys,

    I have one problem that I mentioned in the following

    1) Before merging, we take the videos URLs(static videos or camera captured videos) from array, and store them into assets.

    2) In the processing of storing the URLs into assets,

    1) First, we take the first URL into 'firstVideoAsset', after that we take second URL into 'secondVideoAsset'. Then merge the two videos and store the merged URL into again 'firstVideoAsset'.

    2) After that, take the take third URL into 'secondVideoAsset'. Them merge the two videos. Same process will go on until the loop ends.
    3) Send the final merged URL into MPMoviePlayerController to play the 'preview' of the merged video.

    Orientation issue:-

    1) While in the merging process, if we capture the video in 'landscape' orientation(i.e. LandscapeLeft), the captured video after merging, is coming in 'Upside-down' mode(i.e. reverse of captured video).

    2) The sizes of all the videos(static or camera captured) are not coming in same size after merging.

    3) If we capture the video using front camera, after merging all, the captured videos are coming in upside down.

    Need help from your side:-

    1) Before merging, if we capture the videos in any orientation mode, they have to come in only one orientation(i.e. LandscapeLeft or LandscapeRight) mode after merging.

    2) After merging all the videos, the sizes(height and width) of all the videos(static or camera captured) in any orientation mode, should come in same sizes.

    3) While using front camera, if we capture the video in Landscape left mode, after merging all the videos, the captured videos will come correct mode. But, when capture the video in Landscape right mode, the captured videos after merging all the videos will come in Upside down.(i.e. Reverse of original captured video).

    4) While using back camera, if we capture the video in Landscape left mode, after merging all the videos, the captured videos will come in Upside down.(i.e. Reverse of original captured video) . But, when capture the video in Landscape right mode, the captured videos after merging all the videos will come in correct mode.

    ReplyDelete
  47. Great post. I was checking continuously this blog and I’m impressed! Extremely useful information specifically the last part I care for such info much. I was looking for this certain info for a very long time. Thank you and good luck. online marketing ideas 2012

    ReplyDelete
  48. Right here is the right website for anybody whoo really wants to find out about this topic.
    You realize a whole lot its almost hard to arvue with you (not that
    I personally would want to…HaHa). You certainly put a brand new spikn on a topic which has been discusswd for ages.
    Wonderful stuff, just great!

    my homepage what is the best anabolic steroid

    ReplyDelete