PrePaidGSM.net Forum
Travelsim
PrePaidGSM Home National Operators International Cards Contacts Links


Reply
 
Thread Tools Display Modes
Old
  (#1)
DaveRo (Offline)
Senior Member
Prepaid Fan
 
Posts: 153
Join Date: 07 Jul 2006

Country:
Default Why doesn't WGET work with Cosmote or Wind GR' - 25-08-2009, 13:46

A rather technical question. On my blog about using Greek phones for weather forecasts
Greek Internet Weather Yacht Vigdis
I explain how I try to reduce the data volume when using Poseidon's site.

I wrote a script that uses WGET to download 2 day's weather maps at 6 hour intervals. It works fine over a wifi connection but not over Cosmote or Wind WAP proxied connections.

So for example this doesn't work:
wget http://www.poseidon.hcmr.gr/images/m...db09082515.png

wheras if I type that address into the browser it does. But I then have to save each image - which is tedious.

On Cosmote myview'n'web the wget hangs.
On Wind Plus NS I get a temporary resolution errror.

Any ideas or suggestions? It must be the proxy. (Spoofing the UAS does nothing.)

Dave
   
Reply With Quote
Old
  (#2)
inquisitor (Offline)
Senior Member
Prepaid Professionist
 
inquisitor's Avatar
 
Posts: 1,399
Join Date: 15 Nov 2006

Country:
Default 25-08-2009, 14:42

I understand, that you run the wget command locally (i.e. on your linux machine, which is connected to the internet via the named providers).
Proceeding on this assumption the problem appears pretty simple. http-applications do always send the so-called USER_AGENT parameter revealing it's identity with each http-request.
When you use wget the http requests sent to the server (and passing the network operator's proxy) will contain the USER_AGENT "wget 1.9.2". The proxy will so identify the request as not coming from a handset and so will block it.
What you need to do now, is faking the USER_AGENT. Therefore simply set the user_agent parameter to some mobile phone when executing wget. In the following example you will send the (pretty long) USER-AGENT of a Nokia N95 with your request:
Quote:
wget --user-agent:"Mozilla/5.0 (SymbianOS/9.2; U; Series60/3.1 NokiaN95/11.0.026; Profile MIDP-2.0 Configuration/CLDC-1.1) AppleWebKit/413 (KHTML, like Gecko) Safari/413" http://www.poseidon.hcmr.gr/images/m...db09082515.png
That should do the job.

Depending on which user_agents are blocked by the corresponding operator you could also use the user agent of a desktop browser like Firefox, which would look like this:
Quote:
wget --user-agent:"Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)" http://www.poseidon.hcmr.gr/images/m...db09082515.png
For details on wget parameters, check the documentation: http://www.gnu.org/software/wget/manual/wget.html


terminals: Samsung: Galaxy S5 DuoS (G900FD); BLU: Win HD LTE; Nokia: 1200; Asus: Fonepad 7 ME372CG; Huawei data: E3372, Vodafone R201, K3765, E1762;
postpaid: O2 on Business XL; prepaid: DE: Aldi Talk, Lidl; UK: 3; BG: MTel, vivacom; RU: MTS; RS: MTS; UAE: du Tourist SIM; INT'L: toggle mobile
VoIP: sipgate.de (German DID); sipgate.co.uk (British DID); ukddi.com (British DID); sipcall.ch (Swiss DID); megafon.bg (Bulgarian DID); InterVoip.com
   
Reply With Quote
Old
  (#3)
DaveRo (Offline)
Senior Member
Prepaid Fan
 
Posts: 153
Join Date: 07 Jul 2006

Country:
Default 27-08-2009, 16:18

Thanks for that comprehensive reply, but I already tried that. I just tried again with Cosmote and your suggested User Agent Strings (UAS) and it definitely makes no difference.

Is it likely that these Wap services will sniff for browsers? I've never seen evidence of that.

There are dozens of params for WGET - I have the manpage; any other suggestions?

Edit: I'm using a Nokia N810 tablet tethered to a Nokia N70 phone.

Dave
   
Reply With Quote
Old
  (#4)
inquisitor (Offline)
Senior Member
Prepaid Professionist
 
inquisitor's Avatar
 
Posts: 1,399
Join Date: 15 Nov 2006

Country:
Default 27-08-2009, 17:09

Have you checked, if faking the user_agent does actually take effect? I just found another syntax for changing the user_agent on Google:
Quote:
$ wget -U "user agent" http://example.com/
You should first make sure that the user_agent manipulation works for sure. You should download http://pgl.yoyo.org/http/browser-headers.php while connected through wifi and check if the user_agent has actually been changed.

If the user_agent is actually changed and it still doesn't work through the cell network, they seem to perform some deeper inspection of your requests. Maybe the proxy checks in the EIR (a network element of every GSM-network) what type of phone you use and then only accepts the user_agent of that certain phone. Then you should use only the user_agent of the phone, which is registered on the network (the Nokia N70 in your case):
Quote:
NokiaN70-1/5.0609.2.0.1 Series60/2.8 Profile/MIDP-2.0 Configuration/CLDC-1.1
If that doesn't work either, you should check if you can download that image when you use the native browser of your N70 and/or of the N810. That could help isolating the cause.


terminals: Samsung: Galaxy S5 DuoS (G900FD); BLU: Win HD LTE; Nokia: 1200; Asus: Fonepad 7 ME372CG; Huawei data: E3372, Vodafone R201, K3765, E1762;
postpaid: O2 on Business XL; prepaid: DE: Aldi Talk, Lidl; UK: 3; BG: MTel, vivacom; RU: MTS; RS: MTS; UAE: du Tourist SIM; INT'L: toggle mobile
VoIP: sipgate.de (German DID); sipgate.co.uk (British DID); ukddi.com (British DID); sipcall.ch (Swiss DID); megafon.bg (Bulgarian DID); InterVoip.com
   
Reply With Quote
Old
  (#5)
DaveRo (Offline)
Senior Member
Prepaid Fan
 
Posts: 153
Join Date: 07 Jul 2006

Country:
Default 28-08-2009, 05:46

Yes, I usually use -U; I tried the longer version yesterday.

I'll try checking if the UAS is actually changed next time I find a wifi - may be a week or two. Maybe this version of WGET, compiled for Maemo, is faulty.

I just tried -S: no server response returned (Cosmote).
   
Reply With Quote
Old
  (#6)
DaveRo (Offline)
Senior Member
Prepaid Fan
 
Posts: 153
Join Date: 07 Jul 2006

Country:
Default 29-08-2009, 06:37

I solved it. I need to set the http_proxy environment variable. I assumed that the connection manager would specify the proxy for WGET since the browser picks it up.
   
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
vBulletin Skin developed by: vBStyles.com
2002- ad libitum PrePaidGSM.net