HOW TO: Image transformation with gestures in Xamarin

Nowadays, stickers are a must-have feature for the modern mobile camera applications. They provide the users with the opportunity to share their emotions that they have felt while taking the photo. A heart on a photo with a loved one, a glass of beer on the photo from a Friday’s party or a sticker with the location of the Saturday’s festival – with those we can say much more than with just two or three words.

Typical interactions with the stickers are to scale them up or down, to rotate them or to move them to the preferred spot on the picture. A well-known layout for the app-users is the sticker to be put in a frame, on which the interaction spots or buttons are located.

In this article:

 

What are the challenges behind its development?



There are challenges mostly by Xamarin, which enables the development of Cross-Platform Native Applications.However, there are some challenges for the developers, which cost time and money, although the functions are rather typical nowadays. To be precise, there is no suitable library in Xamarin for this task. A lot should be programmed by hand and problems, that were already dealt with by others, should be solved once more by the developer.

What is missing in Xamarin?


The app-developer should implement a lot, starting with the sticker frame up to the interaction gestures, native in iOS or Android with the so-called Custom Renderers, because the needed features are missing at this time in Xamarin.Forms.

Interaction Gestures

At the moment,  Xamarin.Forms supports only the Tap, Pinch and Pan gestures. Unfortunately, those are not exactly suitable for the sticker interaction. However, what can be done, is to alter the logic behind the available gestures, so that Pan can be used instead of Drag&Drop, for example. The other possibility is to write Custom Renderers und thereby, to implement the gestures native in Android and iOS. Here you can find yourself before an obstacle, as well, when the needed classes are missing in Xamarin.Forms. Then, you can write a converter that transforms a Xamarin.Forms element into an Android one. A third possibility is to use bitmaps and matrices but it should be done separately in Android and iOS. In such case, the developer should keep in mind the differences in the density and in the position between the displayed and actual image, while writing the needed formulas.

We have chosen the first possibility, for it provides a cross-platform solution and is simpler in the case of a Xamarin.Forms project.

For the flip functionality, we have used a TapGestureRecognizer and the implementation itself is rather uncomplicated
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
///
/// Flip the sticker, when the flip button is clicked.
///The clicked button.
///The tap event.
///
async void FlipOnTapGestureRecognizerTapped(object sender, EventArgs args)
{
if (flipDeg == 0)
{
flipDeg = 180;
sticker.RotateYTo(180);
}
else
{
sticker.RotateYTo(0);
flipDeg = 0;
}
}

Figure 2: Flips gesture implementation

Users want to resize der stickers, therefore we have used a PanGestureRecognizer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
///
/// Resizing the sticker.
///
///The clicked button.
///The tap event.
async void OnResize(object sender, PanUpdatedEventArgs e)
{
switch (e.StatusType)
{
case GestureStatus.Running:
var deltaX = e.TotalX - PanContainer.X;
var deltaY = e.TotalY - PanContainer.Y;
var newWidth = PanContainer.Width + deltaX;
var newHeight = PanContainer.Height + deltaY;
scale = Math.Min(Math.Max(Math.Min((newWidth / PanContainer.Width),
(newHeight / PanContainer.Height)), 0.5), 1.5);
PanContainer.ScaleTo(Math.Min(Math.Max(Math.Min((newWidth / PanContainer.Width),
(newHeight / PanContainer.Height)), 0.5), 1.5));
break;

case GestureStatus.Completed:
scale = sticker.Scale;
break;
}
}

Figure 3: Resize gesture implementation

To match the scenery of a picture, a user wants to rotate his stickers, we also used a PanGestureRecognizer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
///
/// Rotating the sticker.
///
///The clicked button.
///The tap event.

async void OnRotate(object sender, PanUpdatedEventArgs e)
{
switch (e.StatusType)
{
case GestureStatus.Running:
rotated = true;
relativeToTransform.RotateTo(e.TotalX);
rotateDeg = relativeToTransform.Rotation;
break;

case GestureStatus.Completed:
rotateDeg = relativeToTransform.Rotation;
break;
}
}


Figure 4: Rotate gesture implementation

For the dragging functionality, we have also used a PanGestureRecognizer. For the implementation, the developer should check if the image has been flipped or rotated and use this information in the formula for the movement
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
///

/// Moving the sticker.
///

///The clicked button. ///The tap event.
void OnMove(object sender, PanUpdatedEventArgs e)
{
switch (e.StatusType)
{
case GestureStatus.Running:
if (flipDeg == 0)
{
if ((rotateDeg > 50 && rotateDeg < 120) || (rotateDeg < -240 && rotateDeg > -310))
{
dx = y + e.TotalY;
dy = x + e.TotalX;
stackPhoto.TranslateTo(-y - e.TotalY , x + e.TotalX , 100);
}else
...
}
}
else //flipped
{
if ((rotateDeg > 50 && rotateDeg < 120) || (rotateDeg < -240 && rotateDeg > -310))
{
dx = y + e.TotalY;
dy = -x - e.TotalX;
stackPhoto.TranslateTo(-y - e.TotalY, -x - e.TotalX, 100);
}
else
...
}
}
break;
case GestureStatus.Completed:
// Store the translation applied during the pan
x = stackPhoto.TranslationX;
y = stackPhoto.TranslationY;
break;
}
}

Figure 5: Drag&Drop (Move) gesture implementation

Sticker Frame

This kind of frame is well-known from a lot of applications and dominates over all other alternaves like the Multi-Touch Gestures. The reason for this is that thanks to the frame, it is clearer to the user how to interact with the sticker and namely – to use the corresponding buttons.

The first idea that comes to a developer in such a situation is to use the Frame class, included in Xamarin.Forms. However, it functions not exactly as needed in this case. Therefore, we have created a renderer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
class CustomFrameRenderer : FrameRenderer
{
protected override void OnElementChanged(ElementChangedEventArgs e)
{
base.OnElementChanged(e);
if (e.NewElement != null)
{
if (!PhotoEditPage.instance.StickerFrame.BackgroundColor.Equals(Color.Transparent))
{
ViewGroup.SetBackgroundResource(Resource.Drawable.shadow);
}
}
}
}

Figure 6: Custom Frame in Android (1)

1
2
3
4
5
6
7
8
9
10
<?xml version="1.0" encoding="UTF-8" ?>
<shape xmlns:android="http://schemas.android.com/apk/res/android" android:shape="rectangle">

  <stroke android:width="1dp" android:color="@android:color/white"></stroke>

  <corners android:topLeftRadius="10dp"
           android:topRightRadius="10dp"
           android:bottomLeftRadius="10dp"
           android:bottomRightRadius="10dp" />
</shape>

Figure 7: Custom Frame in Android (2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
///

/// Calculates the space left on the top and on the left in the
/// ImageView, when an image/photo is displayed.
///

public async void calculateRatioAsync()
{
Bitmap photo = null;
using (var bitmap1use = await
GetBitmapFromImageSourceAsync(PhotoPage.instance.PhotoImage.Source,
MainActivity.context))
{
photo = bitmap1use.Copy(bitmap1use.GetConfig(), true);
}

Image imageView = PhotoPage.instance.PhotoImage;
double bitmapRatio = ((double)photo.Width) / photo.Height;
double imageViewRatio = ((double)imageView.Width) / imageView.Height;

//double drawLeft;
double drawHeight;

// double drawTop;
double drawWidth;

if (bitmapRatio &gt; imageViewRatio)
{
drawLeft = 0;

//Height of the bitmap displayed
drawHeight = (imageViewRatio / bitmapRatio) * imageView.Height;
drawTop = (imageView.Height - drawHeight) / 2;
}
else
{
drawTop = 0;

//Width of the bitmap displayed
drawWidth = (bitmapRatio / imageViewRatio) * imageView.Width;
drawLeft = (imageView.Width - drawWidth) / 2;
}
}

Figure 8: Custom Frame in iOS

Saving the image with the sticker

Here we also have some possibilities to choose from. One is to use once more Custom Renderers and to save the photo together with the sticker as one image by using the classes Bitmap and Canvas. An alternative, which we have chosen, is to make a screenshot of the needed part of the app screen and save it as a bitmap. This should also be done in Android as well as in iOS separately.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
///
/// Calculates the space left on the top and on the left in the
/// ImageView, when an image/photo is displayed.
///

public async void calculateRatioAsync()
{
Bitmap photo = null;
using (var bitmap1use = await
GetBitmapFromImageSourceAsync(PhotoPage.instance.PhotoImage.Source,
MainActivity.context))
{
photo = bitmap1use.Copy(bitmap1use.GetConfig(), true);
}
Image imageView = PhotoPage.instance.PhotoImage;
double bitmapRatio = ((double)photo.Width) / photo.Height;
double imageViewRatio = ((double)imageView.Width) / imageView.Height;
//double drawLeft;
double drawHeight;
// double drawTop;
double drawWidth;
if (bitmapRatio &gt; imageViewRatio)
{
drawLeft = 0;
//Height of the bitmap displayed
drawHeight = (imageViewRatio / bitmapRatio) * imageView.Height;
drawTop = (imageView.Height - drawHeight) / 2;
}
else
{
drawTop = 0;
//Width of the bitmap displayed
drawWidth = (bitmapRatio / imageViewRatio) * imageView.Width;
drawLeft = (imageView.Width - drawWidth) / 2;
}
}


Figure 9: Calculating the left space around the photo in the ImageView (Android).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
var view = Activity.Window.DecorView;
view.DrawingCacheEnabled = true;

DisplayMetrics dm = MainActivity.context.Resources.DisplayMetrics;
float densityDpi = dm.Density;

TabNavPage mainPage = (TabNavPage)App.Current.MainPage;

int statusBarHeight = 0;
int resourceId = MainActivity.context.Resources.GetIdentifier("status_bar_height", "dimen",
"android");
if (resourceId &gt; 0)
{
statusBarHeight = MainActivity.context.Resources.GetDimensionPixelSize(resourceId);
}

Rectangle rect = mainPage.pagesCarousel.Bounds;

int width = Convert.ToInt32(rect.Width * densityDpi - drawLeft * 2* densityDpi);
int height = Convert.ToInt32(rect.Height * densityDpi - drawTop * 2* densityDpi);
int X = Convert.ToInt32(rect.X * densityDpi + drawLeft * densityDpi);
int Y = Convert.ToInt32(rect.Y * densityDpi + statusBarHeight + drawTop * densityDpi);

Bitmap bitmap = view.GetDrawingCache(true);
Bitmap croppedBitmap = Bitmap.CreateBitmap(bitmap, X, Y, width, height);

byte[] bitmapData;

using (var stream = new MemoryStream())
{
croppedBitmap.Compress(Bitmap.CompressFormat.Png, 0, stream);
bitmapData = stream.ToArray();
}

Figure 10: Getting a bitmap Array of the needed area (Android)

Conclusion

A lot of developers ask in the Xamarin forums – why is there still no official library for this, when it is such a widespread functionality. Well, maybe we will see one soon.

And for the time beeing, feel free to use this code and leaf your ideas in the comments!

Tagged on:                         
Sonia Grozdanova

Sonia Grozdanova

Sonia Grozdanova ist unsere Spezialistin für Mobile App Development und moderene Cloud Architekturen.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.