[{"data":1,"prerenderedAt":219},["ShallowReactive",2],{"content-query-8bDOsQvd7b":3},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":10,"date":11,"cover":12,"type":13,"category":14,"body":15,"_type":213,"_id":214,"_source":215,"_file":216,"_stem":217,"_extension":218},"/technology-blogs/en/1847","en",false,"",[9],"AI Design Patterns","The code implementation in Eager mode is simpler than that in Pipeline mode. Therefore, the Eager mode can be used to quickly implement functions when high performance is not required or the data volume is small.","2022-06-01","https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2022/09/29/70b99b7e501a42a2aa2150542b9b2eda.png","technology-blogs","Influencers",{"type":16,"children":17,"toc":202},"root",[18,32,42,47,52,60,65,72,77,84,96,103,108,114,119,126,131,141,146,154,160,165,173,179,183,188],{"type":19,"tag":20,"props":21,"children":23},"element","h1",{"id":22},"ai-design-patterns-04-data-processing-in-eager-mode",[24,30],{"type":19,"tag":25,"props":26,"children":27},"span",{},[28],{"type":29,"value":9},"text",{"type":29,"value":31}," 04-Data Processing in Eager Mode",{"type":19,"tag":33,"props":34,"children":35},"p",{},[36],{"type":19,"tag":37,"props":38,"children":39},"strong",{},[40],{"type":29,"value":41},"[AI Design Patterns] 04-Data Processing in Eager Mode",{"type":19,"tag":33,"props":43,"children":44},{},[45],{"type":29,"value":46},"June 1, 2022",{"type":19,"tag":33,"props":48,"children":49},{},[50],{"type":29,"value":51},"As mentioned in our previous blog, if resources permit, we can use the Pipeline mode to accelerate processing and achieve higher performance. However, with small volume of training data, insufficient training resources, or scattered inference samples, we may fail to apply the Pipeline mode or the code implementation will be complex. In this case, operators for data processing can be directly called to complete data processing in serial mode, which is also known as the Eager mode.",{"type":19,"tag":33,"props":53,"children":54},{},[55],{"type":19,"tag":56,"props":57,"children":59},"img",{"alt":7,"src":58},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2022/09/29/d0eaf3daa1df4dcf9e5c5ddd93d33173.png",[],{"type":19,"tag":33,"props":61,"children":62},{},[63],{"type":29,"value":64},"Overview of AI design patterns",{"type":19,"tag":66,"props":67,"children":69},"h2",{"id":68},"pattern-definition",[70],{"type":29,"value":71},"Pattern Definition",{"type":19,"tag":33,"props":73,"children":74},{},[75],{"type":29,"value":76},"In Pipeline mode, the map operator needs to be defined. It starts and executes the specified data augmentation operator (which can be executed in parallel) to map and change pipeline data. In code implementation, developers need to gradually define operators in each phase of the data pipeline starting from building the input source. The data augmentation operator is involved only when the map operation is performed. For simpler scenarios, the Pipeline mode will increase development burden.",{"type":19,"tag":33,"props":78,"children":79},{},[80],{"type":19,"tag":56,"props":81,"children":83},{"alt":7,"src":82},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2022/09/29/479d8631cd6d4c23ae3bbd1ed824ad3d.png",[],{"type":19,"tag":33,"props":85,"children":86},{},[87,89,94],{"type":29,"value":88},"The ",{"type":19,"tag":37,"props":90,"children":91},{},[92],{"type":29,"value":93},"Eager mode is a lightweight mode for data processing",{"type":29,"value":95},". Developers execute data processing operators by calling functions without building pipelines. Code compilation is simpler and can be executed immediately to obtain running results. It is applicable to lightweight scenarios such as small-scale data augmentation experiments and model inference. As shown in the following figure, compared with the Pipeline mode, the Eager mode needs no pipeline for data processing, making it easier for developers.",{"type":19,"tag":33,"props":97,"children":98},{},[99],{"type":19,"tag":56,"props":100,"children":102},{"alt":7,"src":101},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2022/09/29/b50a0a81344246b8bec78a0092384f44.png",[],{"type":19,"tag":33,"props":104,"children":105},{},[106],{"type":29,"value":107},"Currently, common operators and operators for image/text processing can be executed on MindSpore in Eager mode, which is supported in the vision module (mindspore.dataset.vision), text module (mindspore.dataset.text) and transform module (mindspore.dataset.transforms) on MindSpore.",{"type":19,"tag":66,"props":109,"children":111},{"id":110},"cases",[112],{"type":29,"value":113},"Cases",{"type":19,"tag":33,"props":115,"children":116},{},[117],{"type":29,"value":118},"The following describes how to process image and text data in Eager mode using the MindSpore dataset interface.",{"type":19,"tag":120,"props":121,"children":123},"h3",{"id":122},"image-data-processing",[124],{"type":29,"value":125},"Image Data Processing",{"type":19,"tag":33,"props":127,"children":128},{},[129],{"type":29,"value":130},"The banana image is used as an example for image conversion in Eager mode of MindSpore. First, download the image from OBS.",{"type":19,"tag":132,"props":133,"children":135},"pre",{"code":134},"import wget\n\n\nwget.download(\"https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/datasets/banana.jpg\", \".\")\n",[136],{"type":19,"tag":137,"props":138,"children":139},"code",{"__ignoreMap":7},[140],{"type":29,"value":134},{"type":19,"tag":33,"props":142,"children":143},{},[144],{"type":29,"value":145},"Then, the c_transforms and py_transforms operators in the vision module are mixed to transform the given image. The Eager mode of the vision operator supports the numpy.array or PIL.Image types of data as the input parameter, and pipelines are not required during the process.",{"type":19,"tag":132,"props":147,"children":149},{"code":148},"import numpy as np\n\nfrom PIL import Image\n\nimport matplotlib.pyplot as plt\n\nimport mindspore.dataset.vision.c_transforms as C\n\nimport mindspore.dataset.vision.py_transforms as P\n\n\n\nbanana = Image.open(\"banana.jpg\").convert(\"RGB\")\n\nprint(\"Image.type: {}, Image.shape: {}\".format(type(banana), banana.size))\n\n\n\n# Defines the resize operation and executes it immediately.\n\nsquared_banana = C.Resize(size=(320))(banana)\n\nprint(\"Image.type: {}, Image.shape: {}\".format(type(squared_banana), squared_banana.shape))\n\n\n\n# Defines the CenterCrop operation and understands the execution (the size of the middle part of the image is 280 x 280).\n\nsquared_banana = C.CenterCrop((280, 280))(squared_banana)\n\nprint(\"Image.type: {}, Image.shape: {}\".format(type(squared_banana), squared_banana.shape))\n\n\n\n# The ToPIL() interface converts the NumPy image into a Pillow image to facilitate subsequent padding.\n\nto_pil = P.ToPIL()\n\npadding = P.Pad(40)\n\nsquared_banana = padding(to_pil(squared_banana))\n\nprint(\"Image.type: {}, Image.shape: {}\".format(type(squared_banana), squared_banana.size))\n\n\n\n# Comparison of image processing with and without matplotlib to draw images.\n\nplt.rcParams['font.sans-serif']=['KaiTi']\n\nplt.subplot(1, 2, 1)\n\nplt.imshow(banana)\n\nplt.title(\"original image\")\n\nplt.subplot(1, 2, 2)\n\nplt.imshow(squared_banana)\n\nplt.title(\"processed image\")\n\nplt.show()\n",[150],{"type":19,"tag":137,"props":151,"children":152},{"__ignoreMap":7},[153],{"type":29,"value":148},{"type":19,"tag":120,"props":155,"children":157},{"id":156},"text-data-processing",[158],{"type":29,"value":159},"Text Data Processing",{"type":19,"tag":33,"props":161,"children":162},{},[163],{"type":29,"value":164},"The following example shows how the Eager mode is applied in text processing for word segmentation and type conversion.",{"type":19,"tag":132,"props":166,"children":168},{"code":167},"import mindspore.dataset.text.transforms as text\n\nfrom mindspore import dtype as mstype\n\n\n\n# Defines the word segmentation operation of WhitespaceTokenizer and executes it immediately.\n\ntxt = \"Welcome to Beijing !\"\n\ntxt = text.WhitespaceTokenizer()(txt)\n\nprint(\"Tokenize result: {}\".format(txt))\n\n\n\n# Defines the ToNumber operation and executes it immediately.\n\ntxt = [\"123456\"]\n\nto_number = text.ToNumber(mstype.int32)\n\ntxt = to_number(txt)\n\nprint(\"ToNumber result: {}, type: {}\".format(txt, type(txt[0])))\n\nOutput:\n\nTokenize result: ['Welcome' 'to' 'Beijing' '!'] # Word segmentation result\n\nToNumber result: [123456], type:  # Result of converting a character string to an integer\n",[169],{"type":19,"tag":137,"props":170,"children":171},{"__ignoreMap":7},[172],{"type":29,"value":167},{"type":19,"tag":66,"props":174,"children":176},{"id":175},"summary",[177],{"type":29,"value":178},"Summary",{"type":19,"tag":33,"props":180,"children":181},{},[182],{"type":29,"value":10},{"type":19,"tag":33,"props":184,"children":185},{},[186],{"type":29,"value":187},"References",{"type":19,"tag":33,"props":189,"children":190},{},[191,193],{"type":29,"value":192},"1. ",{"type":19,"tag":194,"props":195,"children":199},"a",{"href":196,"rel":197},"https://mindspore.cn/docs/programming_guide/en/r1.6/eager.html",[198],"nofollow",[200],{"type":29,"value":201},"Lightweight Data Processing",{"title":7,"searchDepth":203,"depth":203,"links":204},4,[205,207,212],{"id":68,"depth":206,"text":71},2,{"id":110,"depth":206,"text":113,"children":208},[209,211],{"id":122,"depth":210,"text":125},3,{"id":156,"depth":210,"text":159},{"id":175,"depth":206,"text":178},"markdown","content:technology-blogs:en:1847.md","content","technology-blogs/en/1847.md","technology-blogs/en/1847","md",1776506105709]